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The  views,  opinions,  and/or  findings 
contained  in  this  document  are  those 
of  the  authors  and  should  not  be  con- 
strued as  an  official  Department  of 
the  Army  position,  policy,  or  deci- 
sion, unless  so  designated  by  other 
documentat ion . 


FOREWORD 


This  document  is  the  result  of  an  in-depth  study  into  the  current  re- 
quirements and  practices  in  acquisition,  use,  and  configuration  management 
of  a Test  Program  Set  (TPS) . The  TPS  includes  application  software  for  the 
Automatic  Test  Equipment  (ATE)  and  the  Interface  Device  (ID)  to  the  Unit 
Under  Test  (UUT) . 

Data  were  compiled  from  all  three  services  and  industry,  with  emphasis 
on  the  total  TPS  life-cycle  considerations.  Procedures  and  policies  that 
have  been  most  effective  in  ensuring  quality  TPS  were  identified  and  were 
compared  with  the  current  CERCOM/CORADCOM  guidelines.  Guidelines  were  then 
developed  that  integrated  proven  quality  policies,  procedures,  and  practices 
suitable  for  the  current  Army  organization. 
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CHAPTER  ONE 

I 

INTRODUCTION 


1 . 1  OBJECTIVE 

These  guidelines  have  been  prepared  to  assist  U.S.  Army  CERCOM/CORADCOM 
agencies  and  their  supporting  organizations  in  establishing  and  implementing 
policies  and  procedures  for  acquisition,  use,  and  configuration  management 
of  Test  Program  Sets  (TPSs) . The  information  provided  is  directed  toward 
managers  responsible  for  specification,  acquisition,  and  acceptance  of  TPS 
in  both  the  readiness  and  the  research  and  development  elements  responsible 
for  TPS  use  and  maintenance. 


1 . 2  SCOPE 

These  guidelines  establish  requirements  for  Test  Program  Set  acquisi- 
tion, use,  and  configuration  control  for  Automatic  Test  Equipment  (ATE). 
Special  attention  in  these  guidelines  has  been  given  to: 

• Requirements  for  TPS  development,  test,  and  documentation 

• TPS  life-cycle  considerations 

• TPS  configuration  management  and  change  control 

• TPS  acceptance 

• TPS  maintenance,  usage,  and  data  collection 
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1.3  REFERENCE  DOCUMENTS 

The  following  documents  relate  to  the  subject  of  this  study: 

• DoD 

••  DoD  5000.29,  26  April  1976,  Management  of  Computer  Resources 
in  Major  Defense  Systems 

••  DoD/AR-70-37 , Configuration  Management 

• Military  Standards 

••  MIL-STD-490 , Specification  Practices 
••  MIL-Q-9858A,  Quality  Program  Requirements 
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••  MIL-S-52779A,  Software  Quality  Assurance  Requirement 

••  MIL-STD-483,  Configuration  Management  Practices  for  Systems, 
Equipment,  Munitions  and  Computer  Programs 

••  MIL- STD- 4 80 , Configuration  Control 

••  MIL-STD-1521A,  Technical  Reviews  and  Audits  for  System  Equip- 
ment and  Computer  Programs 

••  MIL-STD-482A,  Configuration  Status  Accounting 

••  MIL-STD-1519,  TRD  Preparation  Requirements 

••  MIL-STD-1000A,  Engineering  and  Associated  List  Drawing 

••  MIL-STD-785A,  Reliability  Program  for  Systems  and  Equipment 
Development  and  Production 

••  MIL-T-28800A,  Test  Equipment  for  Use  With  Electrical  and 

Electronic  Equipment,  General  Specifications  for 

••  MIL-HDBK-217B , Reliability  Prediction  of  Electronic  Equipment 

••  MIL-STD-470,  Maintainability  Program  Requirements 

••  MIL-STD-471A,  Maintainability  Verification/Demonstration/ 
Evaluation 

••  MIL-T-21200L,  General  Specification  for  Test  Equipment  for  Use 

With  Electronic  and  Electrical  Equipment 

• Army 

••  ECOMR  702-13,  Management  of  Computer  Software  Quality  Assurance 

••  ECOM  PA76-9,  Preparation  of  UUT  Automatic  Performance  Test 
Programs  for  Avionics  Equipment 

••  DESCOMR  702-1,  Quality  Assurance  Program 

••  USASAMSC  STD  490,  Standard  for  ATE  English  Language  Test  Design 
Document 

••  TOAD  PCC116-111076,  ATS  Application  Program  Verification 
Requirements 

••  ECOMR  10-1,  PAD  Charters 

••  AR  702-4,  Procurement  QA 

• Industry  — RCA  Corp.  - Program  Design  Handbook  for  ATE 

• Navy 

••  NAVMATINST  3960.9,  Acquisition  Planning  Guide  for  Automatic 
Test,  Monitoring  and  Diagnostic  Systems  and  Equipment 

••  NAVAIRSYSCOM  AR-9A,  General  Requirements  for  Operational  Test 
Program  Sets 

• Air  Force 

••  ESD  TR-75-91,  Software  Acquisition  Management  Guidebook 

••  ESD  TR-76-365 , An  Air  Force  Guide  to  Contracting  for  Software 
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1.4  ORGANIZATION 


Following  this  introductory  chapter,  the  remainder  of  this  document  is 
organized  into  three  chapters  and  six  appendixes. 

Chapter  Two,  TPS  Acquisition  Guidelines,  is  intended  to  be  used  by 
Government  procurement  personnel  as  part  of  a statement  of  work.  All 
phases  of  TPS  acquisition  are  described  in  detail,  from  development  of 
requirements  to  acceptance  of  TPS  deliverables. 

Chapter  Three,  TPS  Usage  Guidelines,  is  designed  for  the  ATE  managers 
and  discusses  TPS  maintenance,  operational  aspects,  and  organization  and 
training  requirements. 

Chapter  Four,  Configuration  Management  Guidelines,  discusses  the  re- 
quirements for  configuration  control  of  TPSs  and  provides  appropriate  re- 
quirements for  reviews,  documentation,  and  change  control. 

Appendix  A provides  definitions  for  terms  that  are  peculiar  to  auto- 
matic testing  and  TPSs. 

Appendix  B lists  and  defines  the  acronyms  used  in  this  document. 

Appendix  C provides  an  outline  for  a suggested  statement  work  for 
use  by  the  technical  planning  and  procurement  personnel. 

Appendix  D defines  the  data  requirements  for  TPS  acquisition. 

Appendix  E is  a checklist  of  the  steps  to  be  taken  and  the  data  to  be 
provided  to  the  TPS  contractors. 

Appendix  F is  a definition  of  recommended  organizational  responsibili- 
ties for  support  of  TPS  maintenance. 
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CHAPTER  TWO 

I 

TPS  ACQUISITION  GUIDELINES 


This  chapter  provides  guidelines  for  the  Government  to  use  during  the 
acquisition  of  TPS  and  contains  all  phases  of  the  contracted  effort,  in- 
cluding design,  development,  test,  documentation,  and  delivery  of  TPSs. 
These  guidelines  shall  apply  to  all  classes  of  UUTs  including  Line  Replace- 
able Units  (LRUs)  and  Printed  Circuit  Boards  (PCBs) . Special  cases  for 
TPSs  developed  for  all  digital  UUTs  using  digital  simulation  and  automatic 
test  program  generation  aids  are  also  addressed  in  this  section.  Where 
variations  in  UUT  types  (i.e. , analog,  digital,  or  hybrid)  require  separate 
treatment,  those  requirements  will  be  clearly  addressed. 
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2.1  GENERAL  REQUIREMENTS 

The  TPS  consists  of  a hardware  Interface  Device  (ID)  that  connects  the 
UUT  to  the  ATE  and  a computer  program  that  controls  the  ATE  functions  by 
automatically  selecting  and  routing  stimuli  to  the  UUT  and  automatically 
measuring  UUT  response.  The  IDs  may  be  shared  by  multiple  devices.  The 
software  portions  of  the  TPSs  shall  be  given  an  identification  number,  which 
corresponds  to  the  associated  UUT  number. 


2.2  TPS  DEVELOPMENT  PLAN 

The  contractor  shall  generate  a TPS  Development  Plan  (DP)  to  be  ap- 
proved by  the  Government.  The  plan  addresses  schedules  of  activities, 
audits,  and  reviews;  configuration  management  practices;  and  documentation 
requirements.  Interface  device  design  and  TPS  reliability,  maintainability, 
and  test  requirements  shall  also  be  included  in  this  plan,  including  UUT 
and  ATE  station  requirements  and  any  other  facility  and  equipment  require- 
ments for  TPS  quality  control  and  acceptance. 

An  important  part  of  the  TPS  DP  will  be  the  provision  for  compatibility 
with  the  ATE  and  procedures  to  ensure  proper  configuration  control  of  the 
TPS,  handling  of  change  proposals,  and  maintenance  and  updating  of  the  con- 
figuration control  baselines.  The  plan  shall  be  kept  current  throughout 
the  duration  of  the  contract  and  shall  guide  the  contractor  and  Government 
personnel  in  the  scheduling  of  reviews  and  audits,  as  well  as  defining  the 
controls  to  be  employed  by  the  contractor  to  assure  contractual  performance. 
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To  provide  guidance  to  the  contractors,  a discussion  of  the  TPS  devel- 
opment activities  is  presented  in  Section  2.3,  which  identifies  issues  to 
be  addressed  by  contractors  in  their  plan.  However,  contractors  are  en- 
couraged to  develop  a plan  that  is  most  efficient  in  their  own  working 
environment.  Chapter  Four  and  Appendix  D contain  further  details  regard- 
ing the  content  of  the  contractor  DP. 


2.3  TPS  DEVELOPMENT  PHASES 

The  major  phases  in  the  development  of  TPSs  are  illustrated  in  Figure 
2-1.  Most  of  the  activities  are  performed  by  the  TPS  contractor  who  is 
responsible  for  developing  the  TPS.  The  role  of  Government  personnel  is 
to  review  the  analysis,  implementation,  and  tests  and  to  perform  evalua- 
tions for  determining  compliance  with  the  procurement  guidelines  and  satis- 
faction of  contract  intent.  Figure  2-1  applies  when  only  limited  ATE  data 
are  available  to  the  potential  contractors  during  procurement.  Figure  2-2 
applies  when  complete  data  are  available  with  the  procurement  package.  The 
initial  development  phases  are  shortened  accordingly. 

2.3.1  Phase  I — Source  Data  Familiarization 


TPS  development  starts  when  the  contractor  test  design  engineers  are 
familiarized  with  available  data  on  the  UUTs  and  the  proposed  ATE.  For  the 
sole  purpose  of  TPS  development,  the  contractor  will  be  entitled  to  receive 
or  have  timely  access  to  all  Government-owned  UUT  data  applicable  to  his 
project.  From  analysis  of  these  data,  the  test  designer  shall  develop  an 
outline  of  the  approach  for  testing  the  UUT  and  shall  identify  any  incom- 
patibilities between  UUT  performance  requirements  and  ATE  capabilities. 

The  contractor  shall  prepare  the  Conceptual  Test  Design  and  Instruction 
Document  (TDID)  package,  which  includes  a test  requirements  analysis,  pre- 
liminary test  strategy  for  performance  and  fault  isolation,  proposed  ATE/UUT 
interconnection  diagrams,  UUT  failure  profiles,  and  a list  of  potential 
problems  with  recommended  solutions . 

2. 3. 1.1  Test  Requirements  Analysis 

The  contractor  shall  review  the  failure-mode  data  and  perform  a test 
requirements  analysis  for  each  UUT.  The  analysis  shall  define  what  consti- 
tutes a failure  on  the  basis  of  the  UUT's  permissible  operating  conditions 
or  component  tolerances.  The  results  of  this  analysis  shall  be  included 
in  the  initial  submission  of  the  TDID.  The  criticality  and  frequency  of 
each  failure  class  shall  be  addressed  to  guide  the  TPS  design  engineers  in 
their  efforts  for  designing  and  implementing  the  TPSs.  TPS  design  and 
testing  shall  be  planned  to  implement  the  maintenance  concepts  of  the  UUT 
Integrated  Logistic  Support  Plan. 

On  the  basis  of  the  intended  use  of  the  TPS  and  the  maintenance 
concept  formulated  for  the  UUT,  the  Government  will  specify  in  the  contract 
the  extent  of  performance  and/or  diagnostic  testing  and  the  requirements 
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Figure  2-1.  TPS  DEVELOPMENT  PHASES  (LIMITED  DATA  AVAIL 
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for  testing  at  environmental  extremes.  However,  all  TPSs  shall  perform  an 
ID  identification  before  performing  the  tests  on  the  UUT. 
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2. 3. 1.2  Concept  Review 

The  concept  review  is  conducted  to  evaluate  the  TPS  Development  Plan 
and  the  proposed  test  strategy  contained  in  the  conceptual  TDID  established 
by  the  test  engineers  before  development  of  detailed  test  design  data.  The 
Government's  participation  in  this  review,  as  well  as  the  later  detailed 
design  review,  debugging  and  integration,  and  acceptance  testing  phases  (as 
described  in  the  following  subsections),  will  ensure  high  confidence  that 
the  delivered  TPS  will  satisfy  contract  performance  requirements. 

2.3.2  Phase  II  — Detailed  Test  Design  and  Instruction  Document  Development 

Upon  approval  of  the  conceptual  TDID  by  the  Government,  the  contractor 
is  authorized  to  proceed  with  the  design  of  the  test.  A detailed  TDID  shall 
contain  the  sequence  of  tests,  a description  of  the  test  setups  and  adjust- 
ments, and  detailed  ATE/UUT  interconnection  diagrams. 

2. 3. 2.1  ID  Identification  Tests 

The  contractor  shall  develop  ID  identification  tests  to  verify  that 
the  correct  interface  device  is  installed  and  connected  properly.  Failure 
of  this  test  shall  result  in  a message  to  the  operator  identifying  the 
failure  and  termination  point  of  the  test.  A separate  ID  identification 
check  is  required  for  this  test.  This  check  should  be  made  of  a component 
value  that  is  not  a function  of  electrical  current  (e.g.,  at  a resistor 
rather  than  a diode).  "Open"  shall  not  be  used  for  ID  or  ID/UUT  identifica- 
tion. "Short"  will  be  acceptable  where  no  passive  components  are  available. 
Where  possible,  this  check  shall  identify  the  various  revisions  of  the 
basic  part  number.  The  operator  shall  be  instructed  to  identify  the  UUT 
visually.  The  program  shall  then  await  operator  identification  before 
proceeding  to  other  tests.  If  there  is  a possibility  of  test  conditions 
that  could  damage  the  ID,  UUT,  or  the  ATE,  a "safe-to-turn-power-on"  test 
shall  be  performed  first. 

2. 3.2.2  ID  Tests 

IDs  are  considered  to  be  the  same  as  any  other  UUT  and  shall  have  a 
complete  TPS  developed  for  them.  The  software  test  program  for  the  ID  shall 
contain  two  paths  from  which  to  choose:  (1)  a complete  performance  test  of 
the  ID  and  (2)  a survey  test  having  sufficient  detail  to  give  a high  confi- 
dence that  the  ID  is  performing  properly. 

2 . 3 . 2 . 3 Performance  Test  Program 

The  performance  test  program  is  a discrete  set  designed  to  determine 
that  the  UUT  is  performing  within  specified  tolerance  limits.  Performance 
tests  shall  be  sufficiently  detailed  to  ensure  that  defective  units  are  not 
accepted  and  that  acceptable  units  are  not  rejected.  The  performance  pro- 
gram shall  store  and  display  all  relevant  test  instructions  and  test  results 
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needed  for  statistical  analysis  in  accordance  with  Section  2.4.4.  Display 
of  test  instruction  data  may  be  inhibited  by  the  operator  at  his  option. 

Performance  tests  shall  always  start  with  a set  of  static  checks,  con- 
sisting of  various  impedance  measurements  to  test  for  open  or  short  condi- 
tions of  power  supply,  bias  lines,  and  relay  coils. 

For  digital  UUTs,  performance  testing  shall  be  performed  at  operational 
clock  and  signal  rates  where  possible.  Operational  rates  are  those  of  the 
UUT  in  its  normal  operational  environment. 

If  the  test  design  engineers  foresee  a high  probability  of  independent 
failures  in  that  UUT  and  believe  that  continuation  of  the  test  will  produce 
useful  data,  performance  testing  may  continue  after  a failure.  When  multi- 
ple faults  occur,  an  attempt  shall  be  made  to  identify  the  most  probable 
cause  of  the  failure.  A successful  performance  test  shall  result  in  a 
printout  of  the  appropriate  messages  and  an  end  to  the  test.  Following 
detection  of  faults,  the  TPS  may  enter  diagnostic  testing  unless  the  ATE 
operator  indicates  otherwise  through  switch  or  "flag"  settings. 

2. 3.2.4  Diagnostic  Program 

The  diagnostic  program  will  identify  faults,  guide  the  operator  in 
probing  if  necessary,  and  isolate  the  faults  to  the  next  lower  assembly 
level  or  defective  component.  The  Government  will  specify  the  level  of 
fault  isolation  required  and  the  method  of  fault  isolation  (guided  probe, 
multiple  probes,  etc.)  for  each  UUT.  Diagnostic  programs  shall  be  designed 
to  minimize  operator  participation,  including  manual  probing.  If  isolation 
cannot  be  accomplished  below  a cluster  of  piece  parts,  an  analysis  shall  be 
required  for  limiting  fault  isolation  to  such  a cluster,  including  speci- 
fication of  the  cluster  size.  Each  piece  part  of  the  cluster  shall  be 
ordered  in  descending  probability  as  the  potential  cause  of  failure. 

Manual  or  guided  probing  shall  be  kept  to  a minimum.  Diagnostic  testing 
that  requires  the  ATE  operator  to  isolate  the  faults  by  opening  signal 
paths  must  be  separately  identified  and  approved  by  the  Government. 

Multiple  faults  should  not  be  considered  unless  they  are  independently 
detectable  at  the  output. 

2. 3. 2. 5 Alignment  Tests 

Alignment  tests  may  be  part  of  the  performance  tests  when  alignment  is 
a part  of  UUT  performance.  For  example,  alignment  of  a PCB  is  permitted 
during  the  PCB  testing;  however,  when  that  PCB  is  part  of  an  LRU,  a need  to 
remove  and  align  the  PCB  prior  to  completing  the  LRU  test  shall  be  classi- 
fied as  a failure. 

The  range  of  alignments  should  not  be  checked  unless  adjustment  is 
necessary  for  the  UUT  to  function  properly  in  the  next  higher  assembly  level. 
When  such  alignment  is  made,  testing  shall  not  continue  if  the  resulting 
position  of  the  adjustment  device  is  at  one  end  of  its  range. 
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2. 3. 2.6  Accuracy  Requirements 


The  allowance  for  measurement  uncertainty  is  usually  permitted  by 
using  techniques  such  as  averaging  or  root-mean-square.  These  allowances 
will  not  be  necessary  if  the  test  instrument  uncertainty  is  no  greater 
than  one- third  of  the  parameter  tolerance  permitted  for  the  UUT. 

2. 3. 2. 7 Detailed  Design  Review 

At  the  completion  of  this  phase,  a detailed  design  review  shall  be 
held.  This  second  major  review  will  be  conducted  by  the  Government  and 
will  check  the  TDID  technical  content  to  determine  coding  readiness.  The 
Government  technical  representative  will  have  the  authority  for  approval 
of  the  detailed  design. 

2.3.3  Phase  III  — TPS  Generation 


The  actual  coding  of  the  test  sequences  and  the  fabrication  of  the 
interface  device  shall  be  initiated  only  after  Government  approval  of  the 
detailed  TDID.  The  contractor  shall  comply  with  the  Government  standards 
for  coding  and  documentation  as  stated  in  the  contract.  The  TDID  shall 
contain  documentation  for  use  and  maintenance  of  the  TPS,  including  operator 
instructions  for  set-up,  execution,  and  tear-down  procedures;  identification 
of  support  equipment;  and  safety  and  other  precautions. 

2.3.4  Phase  IV  — TPS  Debug  and  Integration 

Before  the  TPS  is  loaded  on  the  ATE,  the  contractor  shall  conduct  a 
review  to  verify  that  the  TPS,  both  software  and  ID,  is  ready  to  go  on 
station.  After  the  TPS  has  been  integrated  with  the  ATE,  the  contractor 
will  check  the  performance  of  the  program  by  deliberately  inserting  known 
faults,  which  have  been  agreed  upon,  into  the  UUT.  Any  errors  encountered 
shall  be  noted  and  appropriate  corrections  made  to  the  TPS.  Records  shall 
be  kept  of  all  fault-insertion  and  testing  activities  in  a certified  log- 
book, which  shall  include  a record  of  the  time  expended  on  the  ATE  or  devel- 
opment stations,  number  and  description  of  faults  inserted  and  isolated, 
corrections  made  to  the  TPS,  and  any  problems  encountered  during  the  tests. 

Government  personnel  will  witness  this  fault-insertion  testing.  Their 
observations  will  form  the  best  historical  data  on  the  TPS  performance  and 
will  affect  the  Government  confidence  in  the  final  product. 

2.3.5  Phase  V - TPS  Acceptance  and  Delivery 

At  the  conclusion  of  the  Debug  and  Integration  Phase,  the  contractor 
shall  formally  notify  the  Government  that  the  TPS  is  ready  for  acceptance 
and  shall  provide  to  the  Government  all  materials  to  be  used  in  the  demon- 
stration. The  Government  will  review  the  submitted  material  for  complete- 
ness and  correctness  and  will  participate  in  the  demonstration,  including 
fault  insertion.  As  set  forth  in  the  TPS  contract,  the  Government  will 
require  the  contractor  to  perform  environmental  testing  to  simulate  the 
intended  deployment  environment  (depot,  general  support,  direct  support, 
etc.).  (See  Subsection  4. 1.1. 2.) 
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2 . 3 . 5. 1 Safety 

In  the  design  of  the  TPS,  the  contractor  shall  identify  all  possible  * 

hazards,  including  high  voltage,  fire,  shock,  temperature  extremes,  mechani- 
cal, and  radiation  (electromagnetic  and  nuclear)  and  shall  minimize  their 
effects.  All  ATE  operations,  maintenance  personnel,  the  UUTs , and  the  IDs 
shall  be  protected  from  these  hazards.  Appropriate  and  adequate  warnings 
shall  be  provided  for  hazard  avoidance.  Messages  shall  be  displayed  to  the 
operators  prior  to  all  hazardous  steps  in  the  test  procedure  clearly  de- 
scribing the  types  of  hazards  involved  and  instructing  the  ATE  personnel, 
step-by-step,  on  how  to  test  hazardous  UUTs. 

2. 3. 5. 2 Security 

Contractors  will  be  advised  of  the  security  classification  of  each  UUT 
and  its  associated  TPS  and  the  appropriate  security  regulations  that  should 
be  followed. 

2. 3. 5. 3 Warranty  (Where  Used) 

The  contractor  shall  warrant  TPSs  for  a period  specified  by  the  con- 
tract. The  warranty  shall  protect  against  failures  subsequent  to  accept- 
ance, as  follows: 

• To  detect  or  diagnose  a fault  specified  in  the  TRD , TDID , or  Test 
Plan 

• To  comply  with  safety,  reliability,  maintainability,  and  design 
requirements,  and  the  requirements  of  the  Test  Plan 

The  Government  will  notify  the  contractor  of  such  failures  and  the 
contractor  shall,  with  Government  approval,  correct  these  failures  at  his 
place  of  business  or  at  Government  sites  in  a timely  manner,  generally 
within  15  days  of  notification.  All  costs  associated  with  these  repairs 
shall  be  borne  by  the  contractor.  The  method  of  acceptance  shall  be  speci- 
fied by  the  Government. 

Where  a unit  of  a particular  configuration  was  not  supplied  to  the 
contractor  for  evaluation  during  development  of  TPS,  those  aspects  of  the 
applicable  portion  of  the  TPS  will  be  exempt  from  this  warranty. 

I 

2. 3. 5. 4 Equipment  Requirements 

The  contractor  shall  identify  in  his  proposal  the  Government-furnished 
equipment  (GFE)  and  related  logistical  support  requirements  for  TPS  devel- 
opment, including  UUT,  ATE,  and  computer  resources.  He  shall  also  identify 
schedule  usage  for  each  item  of  equipment.  The  Government  will  provide 
sufficient  numbers  of  UUTs  for  TPS  development  and  validation  in  a timely 
manner. 

Except  for  the  GFE  specified  in  the  contract,  the  contractor  shall 
be  responsible  for  furnishing  all  necessary  personnel,  facilities,  and 
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equipment  for  planning,  development,  testing,  and  acceptance  of  Test  Program 
Sets. 

2.4  TEST  PROGRAM  SET  DESIGN 
2.4.1  General  Guidelines 


The  test  program  shall  be  functionally  modular  with  separate  indepen- 
dent test  program  modules  performing  individual  or  sets  of  discrete  tests. 
The  test  program,  as  well  as  the  modules  within  each  program,  shall  be  de- 
signed so  that  the  natural  reading  of  the  code  text  shall  follow  the  con- 
trol flow  of  the  test  design  and  the  UUT  functional  organization,  i.e.,  top 
down.  Control  architecture,  interfaces,  and  data  flow  between  the  lower- 
and  higher-level  modules  shall  be  designed  before  the  definition  of  lower- 
level  test  modules.  Modules  shall  have  a minimum  number  of  entry  and  exit 
points  and  shall  be  of  minimum  size. 

A standard  module  format  consistent  with  the  implementation  language 
shall  be  devised  to  facilitate  reviewing,  debugging,  and  servicing.  A 
suggested  module  format  is  name,  header  comments,  macros,  pseudo-operations, 
code,  constants,  local  variables,  and  end.  Interdependencies  of  modules 
that  could  result  in  the  "ripple"  effect  (changes  in  one  module  causing 
errors  in  other  modules)  should  be  minimized;  therefore,  independent  modules 
are  preferred. 

Entry  points  shall  be  provided  to  permit  maintenance  personnel  to  enter 
the  test  procedure  for  performing  selected  groups  of  tests.  Special  atten- 
tion shall  be  directed  toward  exercising  UUT/ID  identification  safety  tests 
and  initializations,  as  appropriate,  at  the  various  TPS  entry  points. 

Whenever  structured  control  elements  are  provided  in  the  ATE  language 
used  to  write  the  TPS,  the  contractor  shall  utilize  structured  programming 
techniques  — "Sequence",  "Do  While",  "If-Then-Else" , and  similar  branching 
techniques  --  and  shall  restrict  the  use  of  unconditional  branch  and  jump 
(GO  TO)  instructions  for  instances  such  as  error  processing  and  abnormal 
exits  from  the  program. 

Sufficient  comments,  including  name,  function,  programmer,  date  ac- 
cepted, and  major  revisions,  shall  be  provided  to  facilitate  TPS  status 
reviews  and  maintenance.  Each  major  subfunction,  test,  or  program  proce- 
dure shall  have  comments  describing  the  function  that  follows. 

Test  programs  should  be  designed  so  that,  as  a goal,  no  more  than  80 
percent  of  the  computer  resources  (storage,  cycle  time,  I/O  capability, 
etc.)  would  be  utilized  at  any  time  during  testing. 

The  program  shall,  at  the  conclusion  of  testing,  electrically  discon- 
nect all  stimuli  and  measurement  devices  from  the  UUT. 
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2.4.2  Interface  Devices  (IDs) 


The  interface  devices,  including  all  cables  and  connectors  required  to 
connect  the  UUT  to  the  ATE,  shall  be  designed  in  accordance  with  MIL-STD- 
785A,  MIL-T-21200,  MIL-T-28800A  Type  II  equipment,  and  the  guidelines  dis- 
cussed in  this  section.  The  number  of  IDs  for  each  UUT  shall  be  kept  to  a 
minimum.  Contractors  are  required  to  perform  an  analysis  to  determine  the 
optimum  number  of  required  IDs  for  testing  a given  number  and  type  of  UUT. 
Depot,  general  support,  direct  support,  storage  and  handling,  and  transpor- 
tation requirements  should  be  considered  in  this  analysis. 

2. 4. 2.1  ID  Design 


ID  design  shall  minimize  operator  set-up  time  and  potential  operator 
errors  during  set  up  and  tear  down.  The  ID  design  shall  not  degrade  any 
UUT  performance  parameters.  With  the  exception  of  transformers,  all  com- 
ponents in  the  ID  shall  be  derated  by  a factor  of  two.  The  contractor 
shall  select  components  for  use  in  the  ID  of  the  same  qualification  level 
as  those  used  in  the  UUT.  The  components  shall  be  mounted  so  that  it  will 
not  be  necessary  to  disassemble  large  portions  of  the  ID  for  servicing  and 
maintenance. 

2. 4. 2. 2 Reliability 

Where  environmental  and  usage  factors  are  required,  IDs  shall  be 
designed  in  accordance  with  the  applicable  portions  of  MIL-STD-785A, 
depending  on  the  complexity  of  the  ID.  Combined  MTBF  for  all  ID  elements 
necessary  to  test  a single  UUT  shall  not  be  less  than  the  number  of  hours 
specified  in  the  contract,  calculated  in  accordance  with  MIL-HDBK-217 . 
Reliability  demonstration  requirements  shall  consist  of  testing  all  IDs 
for  contract-specified  numbers  of  consecutive  failure-free  hours  in  the 
environment. 

2.4.2. 3 Maintainability 

IDs  shall  be  designed  for  testing  on  the  ATE,  using  loop-back  cables 
and  other  shorting  devices  provided  by  the  contractor  as  part  of  the  TPS 
deliverable.  For  simple  passive  IDs  where  continuity  can  be  easily  checked 
by  using  a multimeter,  no  loop-back  devices  shall  be  required.  For  IDs 
containing  active  components,  the  TPS  shall  be  capable  of  fault-isolating 
the  ID,  thereby  treating  the  ID  as  a UUT.  All  adjustments,  alignments,  and 
calibration  requirements  of  the  ID  shall  be  included  in  the  TPS. 

The  contracting  officer  will  specify  the  applicability  of  MIL-STD-471 
to  the  maintainability  demonstration  of  the  ID. 

2. 4. 2. 4 Physical -Mechanical  Considerations 

ID  design  shall  conform  to  the  following: 

• Title  29,  Code  of  Federal  Regulations,  Chapter  XVII,  Paragraph  1010, 
"Occupational  Safety  and  Health  Standards" 
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• MIL-STD-454,  Requirement  1 

• MIL-STD-1472 , Requirements  for  Weight,  "Labeling"  (Paragraph  5.13), 
"Color  Coding  of  Indicator  Lights"  (Table  1) 

The  mechanical  design  shall  be  appropriate  to  tolerate  the  heavy  usage 
expected.  All  connectors  shall  be  in  accordance  with  military  standards 
and  shall  be  provided  with  protective  covers. 

2.4.3  Language 


The  entire  test  program  shall  be  written  in  the  test  language  specified 
by  the  Government.  Advance  approval  will  be  required  for  use  of  any  lan- 
guage other  than  that  specified  for  any  portion,  segment,  or  subroutine  of 
the  TPS.  The  contractor  shall  provide  a detailed  impact  analysis  on  life- 
cycle  cost,  testing,  and  maintenance  when  requesting  permission  to  use  other 
languages.  Moreoever,  requests  for  deviations  shall  be  treated  and  sub- 
mitted as  Engineering  Change  Proposals  (ECPs) . 

2.4.4  Operator  Interface 


Communication  with  the  ATE  operator  shall  consist  of  messages  displayed 
on  the  ATE  printer  and/or  console  display  unit(s).  TPSs  shall  offer  the 
options  of  printing  the  TPS/UUT  identification  numbers,  revision  numbers 
and  date  or  program  compilation,  the  basic  data  of  the  tests  being  performed, 
and  the  failure  condition (s)  or  success  of  these  tests  on  the  logging  device. 
These  options  will  improve  traceability  and  provide  a record  for  assessment 
of  the  test  station  performance.  Additional  messages,  including  both  re- 
sults of  the  measurements  and  calculations,  shall  be  available  to  the  oper- 
ator at  his  option.  Set-up  instructions  shall  be  displayed. 

Repair  and  assembly/disassembly  information,  complex  pictorial  data, 
and  waveforms  shall  be  furnished  in  the  TPS  documentation.  The  displayed 
messages  shall  cite  the  appropriate  test  numbers  in  the  TDID  and/or  test 
program  listing.  When  standard  display  messages  are  provided  by  the 
Government,  the  contractor  shall  use  them  whenever  applicable. 

Instructions  for  probing,  alignment,  and  warnings,  in  addition  to 
cautions  and  switch  setting  actions  on  the  UUT,  shall  be  displayed.  Prob- 
ing instructions  shall  be  available  for  reference  by  the  repair  personnel, 
and  warnings  and  cautions  shall  also  be  displayed. 

2.4.5  Design  Aids 

Contractors  shall  obtain  prior  Government  approval  for  development  of 
software  production  or  programming  aids. 

To  reduce  the  cost  of  TPS  development,  contractors  are  encouraged  to 
use  Government-owned  support  software  including  syntax  checkers,  LASARs, 
pin  list  generators,  and  library  maintenance  programs.  Bidders  shall  iden- 
tify the  Government-owned  support  software  packages  they  will  require  as 
identified  in  Subsection  2. 3. 5.4. 
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2 . 5 TPS  ACCEPTANCE 

TPS  acceptance  will  be  on  the  basis  of  several  evaluations.  Acceptance 
tests  shall  be  conducted  by  using  TPS  deliverable  data  items  and  ATE  equip- 
ment that  are  in  correct  working  order  and  current  calibration  status.  With 
the  exception  of  UUT  and  ATE  documentation,  all  tests  shall  be  conducted 
without  reliance  on  any  documentation  that  is  not  a part  of  the  TPS.  When- 
ever such  equipments  are  available,  acceptance  tests  shall  be  performed  on 
ATE  and  UUTs  other  than  those  used  by  the  contractor  for  TPS  development. 

At  the  Government's  option,  TPS  acceptance  tests  may  consist  of  any  or  all 
of  the  following  tests.  Any  discrepancy  between  the  delivered  TPS  and  any 
requirement  of  the  specifications  shall  be  cause  for  TPS  rejection. 

2.5.1  Performance  Test 


Each  TPS  shall  undergo  a complete  performance  test  sequence  (i.e. , "GO" 
chain)  on  a known  good  UUT.  The  Government  may  reserve  the  right  to  run 
the  "GO"  chain  up  to  four  times  to  demonstrate  repeatability. 

2.5.2  Fault-Insertion  Test 


Government  personnel  will,  on  the  basis  of  TPS  development  data,  deter- 
mine the  type  and  the  number  (up  to  the  maximum  allowed  in  the  contract)  of 
faults  to  be  inserted  in  the  UUT. 

Following  each  fault  insertion,  the  TPS  shall  be  exercised  and  the 
results  documented.  At  the  Government's  option,  the  fault  shall  be  removed 
and  the  TPS  rerun  to  verify  the  proper  UUT  operation.  If  any  faults  are 
not  detected  or  isolated  properly,  the  contractor  shall  take  corrective 
action,  including  revalidation  of  previously  undetected  faults. 

2.5.3  Every  Program  Path  Test 

Every  path  in  the  program  ("GO"  chain,  all  abnormal  processing,  diag- 
nostic, alignment/adjustment,  and  housekeeping  function)  may  be  exercised. 
These  tests  may  be  accomplished  by  inserting  faults  or  otherwise  simulating 
the  abnormal  conditions.  Government  personnel  may,  at  their  option,  require 
up  to  100  percent  of  the  TPS  statements  to  be  exercised  for  correctness  and 
validity. 

2.5.4  Other  Tests 

The  Government  may  require  TPS  acceptance  testing  to  be  performed  in 
its  depot  or  under  environmental  conditions  resembling  those  for  which  the 
TPS  is  intended  to  be  used  and  as  specified  in  the  contract. 

"GO"  chain  and  fault-isolation  routines  and  their  compliance  with 
contractual  requirements  will  be  evaluated.  In  addition,  test  program 
structured  design,  modularity,  and  readability  will  be  reviewed  by  the 
Government  and  their  acceptability  will  be  determined;  operator  interface 
communications  adequacy  will  be  evaluated;  and  safety  features  meeting  the 
appropriate  safety  requirements  stated  in  this  document  will  be  reviewed. 
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2 . 6 DELIVERABLES 
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The  following  subsections  address  two  classes  of  deliverables  that  < 

shall  be  required  --  TPS  usage  deliverables  and  TPS  maintenance  deliverables. 

2.6.1  Usage  Deliverables 

TPS  usage  deliverables  shall  be  provided  for  each  UUT  and  shall  con- 
sist of  the  following: 

• Documentation  shall  be  in  accordance  with  the  contract  data  require- 
ments list  (see  Appendix  D) . ' 

• An  interface  device  (ID)  shall  be  used  for  interfacing  the  UUT  with 
the  ATE,  including  all  cables,  connectors,  and  ID  self-test  cables, 
if  any.  A number  of  TPSs  may  use  the  same  ID. 

• Program  object  code  and/or  intermediate  instructions  shall  be  de-  > 

livered  on  the  medium  (mylar  tape,  magnetic  tape,  cassette,  floppy 

disk,  hard  disk,  etc.)  specified  by  the  Government  and  in  a format 
compatible  with  the  ATE.  The  program  object  code  and/or  intermedi- 
ate instructions  shall  be  complete  with  all  necessary  subroutines 
and  shall  be  ready  for  loading  on  the  ATE.  They  shall  not  require 
any  other  programs  or  software  for  their  execution  other  than  the 
normal  operating  system  and  runtime  software  of  the  ATE  configura- 
tion for  which  the  TPS  was  written. 

2.6.2  Maintenance  Deliverables 

TPS  maintenance  deliverables  consist  of  any  and  all  special  equipments, 
cables,  and  tools  (hardware  or  software)  necessary  for  maintenance  of  the 
TPS.  These  deliverables  do  not  include  general-purpose  equipment  such  as 

multimeters,  oscilloscopes,  etc.,  that  are  generally  found  in  the  Government  ' 

maintenance  centers  and  depots. 

In  addition  to  the  source  decks  and  listings,  all  computer  programs, 
programming  packages,  and  programming  aids  developed  or  procured  at  Govern-  , 

ment  expense  for  planning,  development,  production,  test,  maintenance,  and 

acceptance  of  the  TPS  shall  be  deliverable  items.  Complete  documentation  1 

for  these  software  programs  shall  also  be  deliverable. 

In  addition,  the  contractor  shall  supply  a list  of  all  computer  pro- 
grams used  in  any  of  the  various  stages  of  the  TPS  development  that  are  not 
developed  at  Government  expense.  The  list  shall  identify  each  program  by 
name,  revision  number,  manufacturer,  machine,  and  configuration  (hardware 
and  software) . 
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CHAPTER  THREE 


TPS  USAGE  GUIDELINES 


This  chapter  provides  guidelines  for  the  use  and  maintenance  of  Test 
Program  Sets.  These  guidelines  are  intended  for  use  by  Project  Managers 
and  logistic  support  organizations  and  by  contractors  providing  TPS  usage 
and  maintenance  training  and  support  services. 


3 . 1 OPERATIONS  ORGANIZATION 

The  recommendation  is  made  for  establishing  a centralized  TPS  support 
engineering  organization,  having  a central  core  of  experts  with  the  author- 
ity to  hire  qualified  personnel  from  other  sources  to  augment  their  capa- 
bilities during  peaks.  It  should  consist  of  a TPS  Organic  Maintenance 
Group  and  a TPS  Analysis  Group.  This  organization  could  support  many  pro- 
duction testing  groups,  each  supporting  one  or  more  ATEs  and/or  commodities 
The  functions  of  each  group  including  the  operator  function  are  discussed 
in  Appendix  F. 

The  establishment  of  a single  centralized  TPS  configuration  and  main- 
tenance management  facility,  which  supports  a number  of  ATEs  and  material 
commands,  is  necessitated  by  the  large  number  of  TPSs  that  will  be  in  the 
inventory  in  the  immediate  future.  Primary  benefits  from  this  facility 
will  include  the  following: 

• TPS  Compatibility  with  ATE.  The  management  facility  will  be  manned 
by  a central  core  of  experienced  personnel  familiar  with  ATE  capa- 
bilities to  ensure  that  TPSs  designated  for  a given  ATE  will  not 
exceed  its  performance  limits. 

• TPS  Distribution  Control.  The  facility  will  distribute  TPSs  and 
TPS  changes  to  the  maintenance  installations. 

• Historical  Record  File.  The  information  specified  in  Chapter  Four, 
Subsection  4.1.3,  wi] 1 be  gathered  and  stored  at  this  facility  to 
provide  a single  data  base  for  a TPS  effectiveness  analysis  and 
future  TPS  improvements. 

The  management  facility  will  receive  all  proposed  ECPs  and  software 
change  proposals  and  will  ensure  their  distribution  to  all  affected  organi- 
zations for  response  within  the  specified  time  limits. 


3.2  DATA  COLLECTION  REQUIREMENTS 


A data  base  shall  be  established  and  maintained  by  the  TPS  Analysis 
Group  consisting  of  TPS  quality  and  performance,  development,  and  mainte- 
nance cost  data.  Data  may  be  collected  manually  by  using  data  collection 
forms  or  by  automatically  using  the  ATE  and  management  information  system 
program  outputs.  It  is  recommended  that  data  should  be  collected  and  re- 
duced automatically,  and  only  summaries  of  pertinent  data  files  should  be 
retained  in  the  data  base  — avoiding  the  need  for  handling  and  storing 
huge  volumes  of  raw  data. 

Since  free-form  information  and  comments  are  extremely  difficult  to 
tabulate,  care  should  be  exercised  in  standardizing  the  data  formats  and 
minimizing  comments.  While  standardization  will  not  be  a function  of  the 
TPS  management  organization,  the  terms  used  in  the  data  collection  system 
must  be  clearly  defined. 

3.2.1  Data  Required 

Table  3-1  lists  the  minimum  data  necessary  for  TPS  maintenance  analy- 
sis. Some  data  will  be  provided  by  the  data  forms  for  configuration  man- 
agement addressed  in  Chapter  Four  of  this  document. 

3.2.2  Data  Collection  and  Processing  Tools 

Methods  and  programs  shall  be  developed  to  summarize  and  store  the 
data  collected  in  the  data  base  with  multiple-key  access  facilities  for 
easy  retrieval.  The  developed  system  shall  maintain  sufficient  flexibility 
to  permit  collection  of  different  data  types  from  the  various  generations 
of  ATE  and  TPSs.  It  shall  have  the  capability  to  normalize  or  restructure 
the  data  for  useful  comparison  and  systematic  storage  and  retrieval.  Devel- 
opment of  a "data  definition  language"  for  TPS  data  collection  is  highly 
recommended.  This  language  would  specify  the  formats,  length,  keys,  and 
structure  of  the  data  items  residing  in  the  data  base  and  would  have  the 
facilities  to  create,  update,  and  delete  the  data  items.  Other  features 
such  as  data  editing,  security  for  protection  against  loss  of  data,  and 
access  and  change  authorization  may  also  be  implemented. 

Data  collection  tools  shall  provide  the  facilities  for  reporting  the 
status  and  activities  of  the  data  base,  including  data  base  utilization, 
loading,  total  number  of  records,  and  resource  availability  and  shall  sum- 
marize the  various  file  entries. 


3.3  TRAINING  REQUIREMENTS 
3.3.1  Operator  Training 

Operator  training  shall  consist  of  the  following: 

• High  school  education  or  equivalent 

• Basic  electronics  training,  formal  or  on-the-job 
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Table  3-1 . DATA 

REQUIRED  FOR  MAINTENANCE  ANALYSIS 

Data 

Description 

TPS  ID 

Federal  Part  Number,  other  ID  Number 

TPS  Date 

Date  TPS  was  accepted  and  released  for  use 

Project  and  Material  ID 

Name  or  acronym  of  the  Project  (e.g., 

PATRIOT,  TACFIRE)  and  the  subsystem 
(Radio,  Computer,  Radar) 

ATD  Identifier 

ATE  and  configuration  information 

Language  and  Compiler  ID 

Identification  of  the  compiler,  including 
revision  number 

Simulator/Support  Software 

All  programming  and  simulation  aids  used 
in  TPS  development 

Initial  TPS  Contractor 

Initial  developer  of  TPS  — contractor  or 
Government  organization 

Subsequent  Contracts 

One  entry  for  each  subsequent  contract 
identifying  the  contractor  and  the  date 

Cost 

Cost  of  contracts  with  dates 

Comments 

General  comments  regarding  this  TPS,  if  any 

Program  Size 

Size  of  the  test  program  and  tables  in  bytes 

Number  of  Problems 

Reported 

Number  of  problems  reported  that  have  not 
been  identified  as  invalid  with  identifica- 
tion of  outstanding  problem  reports 

Problem  Reports 

One  entry  per  problem  report  identifying 
errors,  discrepancies,  etc.,  reported 
including  date  reported 

Number  of  Change  Orders 

Number  of  change  orders  already  implemented 

Organic  Changes 

One  entry  for  each  organic  change  request 

TPS  Warranty 

One  entry  for  each  warranty  action  request 

Frequency  of  Usage 

Hours  per  year 

Relationships  With  Other 
TPSs 

Relationships  with  other  TPSs,  including 
sharing  IDs  or  program  subroutines  or 
segments 

Management  Techniques 

Management  techniques  such  as  top  down/ 
structured  design,  and  chief  programmer 
team 

ATE  operator  training,  equivalent  to  the  course  normally  offered  by 
the  ATE  manufacturers 


, • TPS  operations  training  courses  for  each  group  of  TPS  families  or 

j commodities  — these  courses,  whether  offered  by  the  TPS  contractor 

» or  the  Government  should  assure  that  following  its  completion,  the 

operator  will  properly  select  the  TPS  and  the  ID  for  each  UUT  and 
* perform  all  necessary  testing,  as  well  as  conduct  ID  maintenance 

Additional  commodity-peculiar  UUT  training  may  be  required  to  provide 
the  operators  with  sufficient  UUT  knowledge  to  resolve  most  ambiguities  in 
identifying  the  UUT,  ID,  or  the  ATE  as  the  source  of  the  faults.  Operator 
training  periods  should  be  short  (8  to  120  hours)  and  should  be  limited  to 
those  skills  necessary  for  ATE  and  TPS  operational  functions. 

Lead  operators  and  test  supervisors  require  the  same  training  as  the 
operators , in  addition  to  at  least  four  years  of  hands-on  ATE  operation 
experience. 

3.3.2  Engineering  Personnel  Training 

It  is  recommended  that  all  engineers,  programmers,  and  analysts  in 
the  TPS  maintenance  organization  obtain  TPS  development  and  validation 
experience  through  on-the-job  training.  Those  engineering  personnel  who 
have  not  developed  and  validated  several  TPSs  should  be  given  the  opportun- 
ity to  do  so  in  the  Organic  Maintenance  Group  before  joining  the  TPS  Analy- 
sis Group. 
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CONFIGURATION  MANAGEMENT  GUIDELINES 


4.1  GENERAL  REQUIREMENTS 

This  chapter  provides  guidelines  for  management  and  technical  personnel 
who  are  planning,  procuring,  managing,  or  conducting  configuration  control 
of  TPSs.  Configuration  management  procedures  should  be  required  and  docu- 
mented for  the  identification,  control,  updating,  and  accounting  for  the 
status  of  Government-accepted  TPSs.  Specific  requirements  for  a configura- 
tion management  system  are  described  in  this  chapter. 

4.1.1  Configuration  Management  within  TPS  Life  Cycle 

TPSs  are  part  of  an  integrated  ATE-UUT  support  system  consisting  of 
both  hardware  and  software  elements  whose  configuration  should  be  identified 
and  controlled  systematically  and  consistently.  This  management  is  neces- 
sary to  assure  mutual  compatibility  of  elements  during  the  development  and 
production  of  the  TPS,  UUT,  and  ATE.  The  implementation  of  configuration 
identification  and  control  of  all  TPS  elements  is  prescribed  by  the  guide- 
lines addressed  in  the  following  subsections. 

4 . 1 . 1 . 1  Baseline  Descriptions 

A baseline  must  be  established  for  the  control  of  changes  to  TPS  ele- 
ments. Many  direct  interdependencies  are  involved  in  establishing  such  a 
baseline;  but  once  established,  this  baseline,  in  addition  to  subsequently 
approved  changes,  will  identify  the  current  configuration. 

Three  interrelated  baselines  should  be  considered  — ATE,  UUT,  and  TPS. 
When  a TPS  is  developed  for  fielded  equipment,  the  UUT  baseline  has  already 
been  established  by  production  drawings  and  specifications,  and  formal  ECP 
procedures  (in  accordance  with  MIL-STD-480)  should  be  in  effect.  For  UUTs 
that  are  currently  under  development  and  that  will  require  the  availability 
of  TPSs  when  fielded,  there  may  be  no  formal  UUT  product  baseline.  To  mini- 
mize the  effect  of  frequent  UUT  design  changes  on  TPS  development,  it  is 
recommended  that  TPS  development  be  initiated  at  the  latest  possible  date 
commensurate  with  maintaining  schedule  requirements  (e.g. , DT/OT  II  card 
and  module  repair  requirement) . A similar  problem  exists  when  the  ATE  is 
under  development.  It  is  recommended  that  TPS  development  should  not  pro- 
gress beyond  the  conceptual  design  phase.  The  TPS  contractor  may  initiate 
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activity  up  to  the  detailed  design  review  of  the  TPS  without  having  access 
to  an  ATE.  Before  that  review,  the  Government  will  have  determined  that 
the  ATE  design  is  available  and  that  it  will  be  certified  when  the  TPS 
contractor  is  ready  to  initiate  coding. 

Change  control  procedures  and  documentation  for  UUTs  and  ATEs  shall  be 
in  accordance  with  MIL-STD-480.  These  changes  are  described  in  subsequent 
subsections . 

4. 1.1. 2 Reviews  and  Audits 

For  TPS  development,  three  Government  reviews  (as  a minimum)  are  re- 
quired to  assure  adequate  visibility,  early  problem  identification,  and 
control  of  the  contractor's  performance.  During  these  reviews.  Government 
approval  will  be  required  for  selected  data  items,  describing  the  contrac- 
tor's current  TPS  design  (see  Subsection  4.1.2). 

Concept  Review 

The  concept  review  will  be  held  after  the  TPS  contractor  has  formulated 
the  test  strategy  and  has  submitted  to  the  procuring  activity  the  prelimi- 
nary Test  Design  and  Instruction  Document  (TDID)  and  TPS  Development  Plan. 
(See  Appendix  D for  detailed  descriptions  of  data  items.)  During  the  re- 
view, at  least  the  following  items  shall  be  discussed:  functional  descrip- 
tion of  UUT , design  philosophy,  UUT  documentation  problems,  test  require- 
ments analysis,  accuracy  requirements,  use  of  standardized  test,  failure 
profile,  possible  performance  and  diagnostic  tests,  ATE  compatibility  with 
the  test  requirement,  and  preliminary  ID  performance  and  design  requirements. 
In  addition,  basic  software  design  characteristics  (e.g.,  top-down  structured 
code,  available  storage,  memory  maps,  timing  and  sizing  data,  and  opera- 
tional interfaces)  shall  be  reviewed. 

The  information  for  this  review  will  be  contained  in  the  TPS  Design 
Plan  and  the  preliminary  TDID  as  shown  in  Appendix  D.  In  addition,  any 
existing  documents  that  have  been  or  are  planned  to  be  used  by  the  TPS  con- 
tractor shall  be  made  available  for  review  by  the  procuring  activity. 

Detailed  Design  Review 


The  detailed  design  review  will  determine  the  status  of  the  conceptual 
design  and  the  preparedness  of  the  TPS  contractor  to  proceed  with  coding. 
This  review  will  be  held  after  completion  and  submittal  of  the  detailed 
TDID  and  preliminary  TPS  test  plans  and  procedures,  and  it  will  ensure  that 
the  criteria  established  at  the  concept  review  have  been  implemented. 

The  engineering  design  of  the  TPS  ID  hardware  shall  be  presented  to  the 
procuring  activity  in  the  form  of  functional  block  diagrams,  engineering 
drawings,  schematics,  and  possibly  mock-ups.  The  technical  justification 
for  the  number  of  proposed  IDs  shall  be  presented.  The  plan  for  meeting 
the  performance  and  design  requirements  established  during  the  concept 
review  shall  be  indicated. 
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Detailed  English-language  procedures  and  functional  flow  charts  shall  ! 

be  prepared  for  review  to  establish  the  compatibility  of  the  TPS  with  the 
Test  Requirements  Document.  Test  plans  and  procedures  will  be  reviewed 

for  satisfaction  of  the  fault-detection  and  -isolation  requirements  of  the  4 

TPS.  Computer  loading,  processing  time,  and  memory  estimates  will  also 
be  reviewed. 

At  the  successful  completion  of  the  detailed  design  review,  the  Govern- 
ment will  grant  the  TPS  contractor  permission  to  proceed  with  coding. 

Formal  TPS  Acceptance  , 

i 

The  final  TDID,  acceptance  test  plan  and  procedures,  ID  manufacturing  j 

drawings,  source  listings,  and  object  code  will  be  delivered  to  the  Govern- 
ment at  least  15  days  before  the  TPS  acceptance  test.  A meeting  will  be 
held  to  discuss  the  general  guidelines  and  procedures  to  be  used  during 

acceptance  testing  and  to  review  the  delivered  documentation  in  accordance  [ 

with  contract  requirements.  Upon  concurrence  by  all  parties,  a fault  list  1 

shall  be  provided  by  the  procuring  activity  to  the  TPS  contractor. 

Acceptance  by  the  Government  of  the  TPS  will  be  contingent  upon  the 
satisfactory  performance  of  the  TPS  elements  (program,  ID,  and  TDID) . The  ! 

Acceptance  Test  Procedure  (ATP)  shall  define  the  procedures  to  be  followed 
for  demonstrating  the  TPS  capability  to  achieve  fault  detection  and  isola- 
tion to  the  contractually  specified  levels. 

Representative  faults  chosen  by  the  Government  to  encompass  a majority 
of  typical  maintenance  actions  will  be  inserted  into  the  UUT.  Fault  inser- 
tion will  be  witnessed  by  the  procuring  activity  representative,  who  will 
record  the  nature  and  accuracy  of  the  fault  insertion  in  the  ATP  data  sheets. 

A post-demonstration  meeting  shall  be  held  to  discuss  the  Government's 
acceptance  or  rejection  of  the  TPS.  If  the  TPS  is  rejected,  the  alterna- 
tives will  be  discussed  and  a course  of  action  will  be  recommended  by  the 
Government.  The  recommendation  may  include  redesign  or  rework  of  the  TPS. 

4. 1.1. 3 Change  Control  and  Updating 

All  elements  of  TPSs  shall  be  subject  to  formal  engineering  change 
control  procedures  in  accordance  with  MIL-STD-480.  Special  configuration 
control  procedures  for  software-only  changes  shall  be  in  accordance  with 
Section  4.3  of  this  document.  During  the  TPS  development  cycle,  it  is  the 
responsibility  of  the  TPS  contractor  to  use  the  procedures  prescribed  in 
the  TPS  Development  Plan  to  assure  that  the  TPSs  for  the  UUT  and  ATE  systems 
are  correct  when  submitted  for  acceptance  testing.  After  formal  acceptance, 
compliance  with  the  procedures  described  in  Sections  4.2  and  4.3  will  be 
required. 

When  engineering  changes  to  the  UUT  for  which  TPSs  have  been  developed 
create  a new  part  number  for  the  UUT,  an  additional  TPS  must  be  developed 
for  the  new  part  number.  No  TPS  will  be  removed  from  the  TPS  library  for 
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any  specific  UUT  until  all  UUTs  of  that  type  in  the  inventory  become  in- 
active. All  TPS  modifications  shall  meet  all  of  the  requirements  of  these 
guidelines . 


When  the  TPS  contractor  is  authorized  to  implement  modifications  to 
TPSs  that  have  been  accepted  by  the  Government,  the  following  requirements 
for  preparation  shall  apply  in  order  of  preference: 

1.  Utilize  the  existing  ID  without  modification 

2.  Utilize  the  expansion  capability  of  the  ID  without  affecting  its 
capability  to  test  other  UUTs  for  which  it  was  designed 

3.  Develop  a new  ID 

4.1.2  Documentation  Requirements  for  TPS  Life  Cycle 

To  establish  TPS  configuration  control,  a complete  chain  of  documenta- 
tion shall  be  identified  that  determines  the  TPS  requirements  and  design. 

This  chain  starts  with  the  identification  and  control  of  the  UUT  as  a con- 
figuration item  in  accordance  with  MIL-STD-480  and  the  certification  of  the 
UUT's  designated  ATE. 

4. 1.2.1  Test  Requirements  Document  (TRD) 

A TRD  shall  be  developed  specifying  test  requirements  for  a UUT  in 
terms  of  its  functional,  electrical,  and  mechanical  interfaces  and  provid- 
ing a complete  description  of  stimuli,  loading  requirements,  and  responses. 
This  document  shall  be  written  by  the  UUT  contractor  for  approval  by  the 
Government. 

After  approval  of  the  TRD,  the  formal  UUT  baseline  for  TPS  development 
shall  be  established.  The  UUT  source  documents  shall  be  appended  to  the 
baseline.  Change  control  procedures  of  MIL-STD-480  shall  be  observed  to 
ensure  that  the  TRD  represents  the  current  UUT  configuration. 

MIL-STD-1519  provides  detailed  guidance  for  the  preparation  of  this 
document. 

4.1. 2. 2 TPS  Development  Plan 

As  part  of  his  proposal  response,  the  TPS  contractor  shall  submit  a 
preliminary  TPS  Development  Plan  (DP) , which  will  be  completed  within  one 
month  after  contract  award  and  submitted  for  final  approval  by  the  Government. 

The  DP  is  the  overall  plan  for  developing  the  TPSs  and  specifies  neces- 
sary supporting  resources,  including  identification  of  the  end  items  to  be 
delivered,  the  schedule  for  each,  and  the  related  documentation  submittals. 

It  shall  include  details  of  the  development  organization  and  assign  respon- 
sibility for  the  design,  implementation,  testing  and  integration,  hardware 
and  facilities  requirements,  and  procedures  for  managing  and  controlling 
the  quality  of  all  aspects  of  TPS  development. 
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The  TPS  contractor  shall  observe  the  procedures,  controls,  and  methods 
specified  in  the  DP,  but  it  must  accommodate  changes  in  schedules  and  per- 
sonnel requirements  during  the  development  period.  Thus  the  TPS  contractor 
shall  be  directed  [through  the  Statement  of  Work  (SOW) ] to  update  the  plan 
as  appropriate. 

The  plan  shall  be  used  to  define  the  control  procedures  for  managing 
design  changes  before  the  establishment  of  baselines  (especially  the  appro- 
val of  the  Test  Design  and  Instruction  Document) . During  ATE  integration 
and  fault-insertion  testing,  the  plan  shall  address  the  reporting  and  man- 
agement of  discrepancies  discovered  in  testing,  responsibilities  for  fail- 
ure analysis  and  correction,  retesting,  and  control  of  both  the  TPS  source 
and  object  code.  An  example  of  a detailed  content  outline  for  the  TPS  DP 
is  contained  in  Appendix  D,  Attachment  2. 

4 . 1 . 2 . 3 Test  Design  and  Instruction  Document  (TDID) 

The  TDID  and  subsequent  updates  shall  be  delivered  to  the  Government 
at  three  distinct  milestones  during  TPS  development  cycle  — before  the 
concept  review,  at  the  detailed  design  review,  and  before  final  acceptance 
testing.  The  content  and  level  of  detail  contained  in  the  submittals  shall 
be  appropriate  to  the  intended  review,  as  detailed  in  the  following 
paragraphs. 

Conceptual  Design  Phase 

The  first  submittal  of  the  TDID  to  the  Government  shall  be  within  fif- 
teen working  days  before  the  concept  review.  At  least  the  first  three  sec- 
tions of  the  TDID  — Introduction,  Test  Requirements  Analysis  (TRA) , and 
Pretest  Data  — shall  be  completed. 

The  major  activity  for  the  TPS  contracter  before  conceptual  design 
will  be  to  analyze  testing  requirements,  to  formulate  a program  design  ap- 
proach for  all  TPSs  that  will  be  developed,  and  to  include  this  information 
in  the  TRA  section  of  the  TDID.  The  TRA  section  shall  consist  of  but  not 
be  limited  to  the  following  information: 

• The  results  of  an  analysis  of  each  UUT  to  determine  requirements 
for  test  stimuli  and  responses,  and  loading,  handling,  warm-up,  and 
interface  connection  requirements 

• An  assessment  of  any  additional  Government-supplied  UUT  test  de- 
scriptive documentation  (schematics,  wiring  diagrams,  assembly 
drawings,  parts  lists,  DMWRS , product  specifications)  required  as 
a basis  for  test  program  design 

• The  anticipated  number  of  functional  and  diagnostic  tests,  together 
with  an  estimate  of  the  level  of  unambiguous  fault  isolation  that 
may  be  achieved 

• A list  of  the  basic  test  groups  and  associated  rationale  for  per- 
forming these  tests 

• Identification  of  unique  adapter  requirements 
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Government  approval  of  the  conceptual  design  will  be  required  before 
proceeding  to  the  detailed  design  phase. 

Detailed  Design  Phase 

The  second  submittal  of  the  TDID  shall  be  within  ten  working  days  be- 
fore the  detailed  design  review.  All  major  sections  of  the  document  shall 
be  submitted  at  this  time,  including  any  revisions  to  the  previous  submittal. 

A detailed  description  of  the  UUT  test  approach  shall  be  provided  in 
an  English-language  descriptive  format.  The  description  shall  include  the 
various  test  groups  and  tiers  of  testing  and  shall  describe  the  implementa- 
tion of  these  tests  and  any  special  requirements.  All  the  parameters  nec- 
essary for  the  UUT  to  meet  its  specification  shall  be  tested,  and  all 
specified  fault-isolation  requirements  of  the  TRD  shall  be  met.  Appendix  D 
provides  an  example  of  a detailed  content  outline.  Upon  approval  of  the 
detailed  TDID,  the  TPS  contractor  may  proceed  with  the  coding  effort. 

Final  TDID 


Before  formal  acceptance  testing  and  after  internal  verification  and 
validation  testing  has  been  completed,  the  updated  final  TDID  reflecting 
the  design  of  the  TPS  program  to  be  used  during  acceptance  testing  will  be 
submitted.  The  only  major  item  of  the  TDID  that  would  not  have  been  re- 
quired with  previous  submittals  would  be  the  program  listings  in  object 
and  source  code. 

Upon  approval  by  the  Government,  the  final  TDID  shall  provide,  for 
example,  language,  flow  charts,  diagrams,  program  listings,  ATE  resource 
requirements,  and  program  start  and  maintenance  procedures,  so  that  a 
programmer  can  locate  all  program  listings,  data  descriptions,  and  system 
and  utility  features  required  to  follow,  understand,  and  maintain  the  TPS 
program. 


4. 1.2. 4 Interface  Device  Development  and  Product  Specifications 

A hardware  specification  in  accordance  with  contractual  requirements 
shall  be  prepared  and  kept  current  by  the  contractor.  The  contractor  shall 
comply  with  the  requirements  of  MIL-STD-480  with  respect  to  changes  to  the 
functional  and  product  baseline. 

4 . 1 . 2 . 5 TPS  Acceptance  Test  Documentation 

Draft  test  plans  and  procedures  shall  be  submitted  ten  days  before  the 
detailed  design  review.  Formal  acceptance  test  plans  and  procedures  shall 
be  prepared  and  delivered  by  the  TPS  contractor  to  the  Government  one  month 
before  acceptance  testing. 

The  plans  and  procedures  shall  establish  the  operating  ground  rules 
for  all  personnel  participating  in  or  supporting  the  acceptance  testing 
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effort.  The  rules  shall  contain  specific  instructions  regarding  ATE  and 
related  test  equipment  for  the  following: 

• Insertion  of  faults 

• Performance  tests 

• Correction  of  program  errors 

• Correction  of  manual  procedures 

• Correction  of  ID  failures 

Upon  successful  completion  of  the  acceptance  test,  the  contractor  shall 
document  the  results  of  all  performance  and  diagnostic  tests  in  an  accept- 
ance test  report.  The  report  shall  also  contain  the  contractor's  test  re- 
sults of  in-house  fault  insertion  and  a log  of  acceptance  activities  to 
form  a TPS  data  history. 

4. 1.2. 6 TPS  Software  Program-Medium 

In  accordance  with  Section  IX  of  the  Armed  Services  Procurement  Regu- 
lations (ASPR)  (as  described  in  Defense  Procurement  Circular  DPC  74-3, 

29  November  1974) , software  shall  be  procured  as  a data  item.  The  TPS 
contractor  shall  deliver  object  and  source  codes  to  the  Government  in  ac- 
cordance with  Appendix  D,  Attachment  2. 

4.1.3  Historical  TPS  Reference  Document 

Figures  4-1,  4-2,  and  4-3  illustrate  the  data  that  shall  be  collected 
for  each  UUT,  ID,  and  ATE  within  the  inventory  to  provide  the  historical 
data  base  for  TPS  studies  and  analysis  as  well  as  change  proposal  evaluation. 

The  records  shall  be  maintained  and  updated  by  the  maintenance  facility. 
During  the  development  stages,  the  TPS  and  ATE  contractors  shall  maintain 
current  data  for  their  specific  tasks.  After  acceptance  by  the  Government, 
the  maintenance  facility  will  keep  the  records  current. 

4.2  HARDWARE-AFFECTED  CHANGE  CONTROL 

All  ECPs  that  affect  ATE  or  UUT  hardware  shall  be  processed  in  accord- 
ance with  MIL-STD-480.  For  UUTs  that  are  tested  on  ATE,  in  addition  to 
meeting  the  requirements  of  MIL-STD-480  for  completion  of  ECPs,  the  origi- 
nator shall  include  the  cost  and  schedule  impact  of  the  ECP  on  the  TPS  and 
ATE. 


4.3  SOFTWARE  CHANGE  CONTROL  GUIDELINES 

Software  change  control  guidelines  apply  to  modifications  to  test 
program  software  and  documents  after  Government  acceptance  that  are  not 
the  results  of  hardware  (UUT  or  ATE)  changes.  All  modifications  shall  be 
performed  in  accordance  with  MIL-STD-480  and  AR-70-37. 
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UUT  Data 


Part  Number: 

Nomenclature : 

Manufacturer's  Name: 

System  Used  In : 

Source  Data 

Assembly  Drawing  Number:  Rev 

Schematic: Rev 

Parts  List: Rev 

Wiring  Diagram: Rev 

TRD : Rev 

TPS  Data 

TPS  DP: Rev 

TDID : Rev 

Program  Media: Rev 

ID  Development 

Specification: Rev 

ID  Production 

Specification : 

ID  Part  Number: Rev 

ATE  Data 

Compiler: Rev 

System  Software  : Rev 

Basic  Building  Blocks: 


Historical 

Trouble  Reports 


Date/Cause/Effect 


ID  Data 

Part  Number: 

Nomenclature : 

Manufacturer's  Name: 

UUTs  Supported: 

Test  Programs  Supported: 

Source  Data 

Assembly  Drawing  Number: 

Rev 

Schematic .- 

Rev 

Parts  List: 

Rev 

Wiring  Diagram: 

Rev 

Development  Specification: 

Rev 

Production  Specification: 

Rev 

ATE  Data 

Compiler : 

System  Software : 

Basic  Building  Blocks: 

Historical 

Trouble  Reports : 

Date/ Cause/Effect 

Figure  4-2.  ID  RECORD 

Change  processing  is  initiated  by  receipt  of  program  or  documentation 
problem  reports,  program  enhancement  requests,  or  waiver  requests.  The 
conf iguration  control  board  shall  review,  identify,  and  recommend  a list  of 
changes,  together  with  itemized  costs  (costs  per  change  or  costs  for  a number 
of  related  changes)  and  pertinent  analysis  or  reasons  for  each  change  and 
testing  required  to  validate  the  changes  where  implemented.  ECPs  will  be 
initiated  if  necessary. 

As  many  changes  as  possible  that  affect  the  same  TPS  shall  be  batched 
to  reduce  the  implementation  and  administrative  costs.  Change  requests  and 
cost  quotations  shall  include  all  necessary  updates  to  TPS  documentation. 

Following  implementation  of  the  changes,  the  Government  will  partici- 
pate in  validation  and  verification  testing  as  required.  Upon  successful 
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ATE  Data 

Part  Number: 

Nomenclature : 

Manufacturer's  Name: 

Building  Blocks: 

UUTs  Supporte  d : 

TPSs  Supported: 

Source  Data 

Assembly  Drawing  Number: 

Rev 

Schematic : 

Rev 

Parts  Lists : 

Rev 

Wiring  Diagram: 

Rev 

Building  Block  Data: 

Rev 

Drawing : 

Rev 

Schematic : 

Rev 

Parts  List-. 

Rev 

Software  Data 

Compiler: 

Rev 

Assemblies : 

Rev 

Operating  System: 

Rev 

Historical 

Trouble  Reports : 

Date/Caus  e/Ef  f e ct 

Figure  4-3.  ATE  RECORD 


verification,  the  Government  representative  will  prepare  a verification 
letter  for  submittal  with  the  developed  changes  and  change  instructions. 

As  a result  of  the  changes,  a kit,  subroutine,  and/or  newly  compiled  pro- 
grams in  the  appropriate  medium,  and  new  documentation  and  change  instruc- 
tions shall  be  delivered. 
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APPENDIX  A 
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DEFINITIONS 

] 

1 

Accuracy.  The  degree  to  which  a measured  value  agrees  with  the  true  value. 

Automatic  Test  Equipment  (ATE) . Equipment  that  is  designed  to  analyze 
static  or  functional  parameters  of  the  units  under  test  (UUT)  to  evaluate 
performance  and  that  may  be  designed  to  indicate  and  isolate  faults  that 
cause  malfunctions  in  the  UUT.  The  decision-making,  control,  and  evalua- 
tions are  conducted  with  a minimum  of  reliance  on  human  intervention. 

Fault.  A failure  causing  a deviation  in  the  specified  functional  perform- 
ance of  a UUT. 

Interface  Device  (ID) . A device  or  series  of  devices  designed  to  provide 
a compatible  connection  between  the  unit  under  test  and  the  automatic  test 
equipment.  An  ID  may  include  special  stimuli  or  loads  that  are  not  con- 
tained in  the  test  equipment. 
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Line  Replaceable  Unit  (LRU) ■ A unit  designed  in  accordance  with  a mainte- 
nance plan  to  be  removed  at  the  organizational  level  upon  failure  from  a 
larger  entity  (equipment,  system,  etc.)  in  the  latter's  operational 
environment. 

Stimulus . Any  input  applied  to  a device  intended  to  produce  a measurable 
response. 

Test  Accuracy  Ratio  (TAR) . The  ratio  of  ATE  accuracy  to  tolerance  limits 
required  for  simulation  and  measurement  for  the  UUT. 

Test  Program  Set  (TPS) . All  elements,  including  hardware,  software,  and 
documentation,  necessary  to  permit  a unit  under  test  to  be  tested  by 
automatic  test  equipment. 
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ATE  Automatic  Test  Equipment 

ATLAS  Abbreviated  Test  Language  for  All  Systems 

CCB  Configuration  Control  Board 

Cl  Configuration  Item 

DID  Data  Item  Description 

DP  Development  Plan 

DS  Direct  Support 

ECP  Engineering  Change  Proposal 

GFE  Government-Furnished  Equipment 

3S  General  Support 

ID  Interface  Device 

ILSP  Integrated  Logistic  Support  Plan 

I/O  Input/Output 

LASAR  Logic  Automated  Stimulus  and  Response 

LRU  Line  Replaceable  Unit 

MTBF  Mean  Time  Between  Failures 

MTTR  Mean  Time  To  Repair 

PCB  Printed  Circuit  Board 

PM  Product  Manager  or  Project  Manager 

QA  Quality  Assurance 

RF  Radio  Frequency 

RFP  Request  for  Proposal 

SOW  Statement  of  Work 

TAR  Test  Accuracy  Ratio 

TDID  Test  Design  and  Instruction  Document 

TPS  Test  Program  Set 


B-l 


TRA 


Test  Requirements  Analysis 
Test  Requirements  Document 
Unit  Under  Test 


TRD 

UUT 
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SUGGESTED  OUTLINE  FOR  STATEMENT  OF  WORK 


1.  Objective 

2 . Scope 

3.  Background 

4.  Tasks/Technical  Requirements 

4.1  Design,  Fabricate,  and  Deliver 

4.1.1  Design 

4. 1.1.1  General  Design  Requirements 

• TPS  Development  Plan 

• Safety 

• Equipment  Requirements 

• Discrepancies 

• Language 

• Coding  Convention 

• Operator  Interface 

• Support  Software 

• Design  Aids 

4. 1.1. 2 TPS  Design 

• General  Guidelines 

• UUT  Testing  Requirements  (TPS  Design) 
••  ID/UUT  Identification  Test 

••  ID  Stimuli  Tests 
••  UUT  Performance  Tests 
• • UUT  Diagnostic  Tests 
••  UUT  Alignments 
• • Accuracy  Requirements 
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• Interface  Device  Design 

• • Performance  Requirements 
• • Reliability 
••  Maintainability 

••  Physical/Mechanical  Considerations 

4.1.2  Fabrication 

4. 1.2.1  Hardware  Fabrication 

4. 1.2. 2 Software  Production 

4.1.3  Deliverable  Items 

4. 1.3.1  For  Each  UUT 

• Test  Program  Object  or  Intermediate  Code 

• Interface  Device 

• TDID 

4. 1.3. 2 Other  Documentation  IAW  CDRL 

4. 1.3. 3 TPS  Maintenance 

• Source  Decks  and  Listings 

• Support  Software 

4 . 2 Program  Management 

4.3  Test  and  Evaluation 

4.3.1  Internal  Test  and  Evaluation 

4.3.2  TPS  Acceptance  Test 

4. 3. 2.1  Performance  Test 

4. 3. 2. 2 Fault-Insertion  Test 

4. 3. 2. 3 Program  Path  Test 

4. 3.2.4  Run-Time  Resource  Measurements 

4. 3. 2. 5 Environmental  Test 

4.4  Data  Management 

4.5  Configuration  Management 

4.5.1  Baseline  Description 

4.5.2  Reviews  and  Audits 

4.5.3  General  Change  Control  and  Updates 

4.5.4  Status  Accounting 

4.5.5  Baseline  Documentation  Requirements 

4. 5.5.1  TDS 

4.5. 5.2  TPS  DP 


4. 5. 5. 3 TDID 

4. 5. 5. 4 ID  Development  and  Product  Specification 

4.5. 5. 5 Acceptance  Test  Plans,  Procedures,  and  Reports 

4.5.6  Centralized  TPS  Management 

4. 5.6.1  Historical  TPS  Reference  File 

4 . 5 . 6 . 2 Hardware  Change  Control 

4. 5. 6. 3 Software  Only  Change  Control 

4.6  Quality  Assurance  Program 

4.7  Warranty 

4.7.1  One-Year  Warranty 

4.7.2  Long-Term  Warranty 
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DATA  REQUIREMENTS 


1 .  PREFACE 

This  appendix  lists  and  briefly  describes  the  minimum  set  of  data 
items  required  by  the  Government  from  the  TPS  contractor.  Existing  Data 
Item  Descriptions  (DID)  have  been  used  to  the  maximum.  Minor  modifications 
may  be  required  for  some  of  the  data  items  to  reflect  specific  Government 
requirements.  For  three  data  items,  new  DIDs  are  required;  their  contents 
are  listed  in  Attachments  1,  2,  and  3 to  this  appendix.  A list  of  addi- 
tional data  items  that  may  be  procured  is  also  provided. 


2.  REQUIRED  DOCUMENTATION 

• DI-E-XX  - Test  Design  and  Instruction  (see  Attachment  1 for  DID) 

• DI-E-YY  - TPS  Development  and  QA  Plan  (Attachment  2) 

• DI-E-ZZ  - Test  Program  Set  Software  Code  (Attachment  3) 

• UDI-E-20064  - ID  Development  Specification 

• DI-E-3132  - ID  Product  Specification 

• DI-T-3714A  - Acceptance  Test  Procedures 

• DI-T-3718A  - Acceptance  Test  Report 

• DI-E-3128  - Engineering  Change  Proposals 

3.  EXAMFLES  OF  OTHER  POSSIBLE  DATA  ITEMS 

• DI-A-5032  - Project  Status  Reports 

• DI-A-3009  - Program  Milestones 

• DI-A-3027  - Data  Accession  List/Internal  Data 

• DI-E-3127  - Advance  Change/Study  Notice 

• DI-E-3145  - Engineering  Drawings  for  Design,  Reviewing,  Audits,  and 

Evaluation 

• DI-F-6004A  - Contract  Fund  Status  Report 
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DI-H-3258A  - Training  Support  Date 

DI-H-6131  - Training  and  Training  Equipment  Plan 

DI-H-6153  - Technical  Manuals/Commercial  Literature 


ATTACHMENT  1 


I 

I 

I 

I 

I 

I 

I 


DI-E-XX 


Block  1.  Test  Design  and  Instruction  Document 

Block  3.  To  specify  the  complete  procedures  and  test  steps,  sequence  of 

tests,  and  test  parameters  required  to  isolate  faults  and  verify 
proper  operation  of  a particular  item  that  is  to  be  tested  on 
automatic  test  equipment.  English-language  statements  are  used 
in  addition  to  flow  charts.  The  TDIDs  will  also  contain  the 
complete  test  requirements  analysis,  program  start-up,  and 
operator  actions,  and  the  program  listing  in  the  ATE  higher- 
order  language. 

Block  10.  See  Addendum  I attached  to  this  DID. 
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1.  SCOPE 

This  addendum  addresses  the  preparation  of  Test  Design  and  Instruction 
Documents  for  systems  supported  by  an  Automatic  Test  Set.  Each  UUT  TDID 
shall  contain  complete  test  (automatic  and  manual)  and  fault-isolation  pro- 
cedures, as  well  as  a test  requirement  analysis,  pre-  and  post-test  proce- 
dures, and  source  program  listings.  Each  TDID  shall  contain  functional  and 
diagnostic  flow  charts  with  English-language  test  descriptions  that  are  all- 
inclusive  and  in  a format  directly  applicable  to  the  coding  and  preparation 
of  a test  program  and  for  qualification  test  and  fault  isolation  of  a spe- 
cific UUT.  Each  TDID  shall  contain  the  interface  requirements  between  UUT 
and  ATE  and  shall  cite  the  interface  device  development  and  product 
spec i f ic  at ions . 


2.  RULES  FOR  PREPARATION  OF  A TEST  DESIGN  AND  INSTRUCTION  DOCUMENT 
2 . 1 Elements  of  the  Document 

A Test  Design  and  Instruction  Document  shall  contain  document  elements 
in  the  following  order: 

• Title  Page 

• Summary  Page 

• Revision  Sheets 

• Table  of  Contents 

• Section  1.  Introduction 

••  UUT  Description  and  Differences 
• • Purpose  and  Scope  of  Test 
• • Programming  Rules 
••  Source  Information 
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IX 


• Section  2.  Test  Description  and  Instructions 
• • Pretesting  Data 

• • • ATA  Resource  Required 
• • • Support  Equipment  Required 
• • Test  Requirements  Analysis 
• • Program  Start  Procedure 
• • Operator  Actions 

••  English-Language  Test  Description 
• • • General 

...  Test  Design  Criteria 
• • • Static  Tests 
• • • Dynamic  Tests 
•••  UUT  Interface  Requirements 
• • Supplementary  Data 
•••  Listing 

• • • Functional  Flow  Charts 
• • • Diagnostic  Flow  Charts 
• • • Interconnection  Diagrams 
•••  Cross-Reference  Table 
•••  Digital  Test  Data 

• Section  3.  Engineering  Notes 

• Section  4.  Quality  Assurance 

Each  of  these  elements  is  described  as  follows: 

• Title  Page.  Figure  1 illustrates  a title  page  for  a Test  Program 
Design  and  Instruction  Document. 

• Summary  Page.  Figure  2 illustrates  a summary  page  for  a Test 
Program  Design  and  Instruction  Document. 

•*  UUT  Number.  UUT  identification  numbers  shall  be  assigned  by 
the  TPS  contractor  and  shall  be  available  on  request. 

••  Date.  Record  the  date  when  manuscript  typing  is  initiated, 
which  will  provide  a base  reference  for  subsequent  revisions. 

• • Number  of  Pages.  Record  the  total  number  of  pages  in  the 
original  manuscript.  This  number  will  change  as  pages  are 
later  revised. 

••  UUT  Nomenclature.  For  each  UUT,  record  the  complete  military 
nomenclature.  If  military  nomenclature  has  not  yet  been 
assigned,  use  the  assembly  common  name. 
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••  Military  Designation  Number.  Record  the  Army  Part  Number  for 
the  UUT.  If  an  Army  Part  Number  has  not  been  assigned,  record 
the  control  number  or  the  manufacturer's  designation. 

••  System.  Indicate  the  system  or  subsystem  of  which  the  UUT  is 
an  integral  part.  Use  official  military  nomenclature  when 
available. 

•*  UUT  Serial  Number  Applicability.  Indicate  the  serial  number 

and  model  of  the  UUT  used  in  the  preparation  of  the  Test  Design 
and  Instruction  Document.  In  addition,  indicate  the  applica- 
bility to  other  UUT  serial  numbers  and  models. 

• • UUT  Serial  Number  at  Validation.  Record  the  serial  number  or 
serial  number  range  of  the  UUT  to  which  this  Test  Design  and 
Instruction  Document  applies. 

*•  Test  Program  Status.  Record  test  program  milestones  such  as 
date  of  design,  validation  sign-off,  and  complete  revision 
sign-offs . 

••  Revision  Sheets.  Revisions  to  a Test  Design  and  Instruction 
Document  shall  be  recorded  on  a status  table  and  revision 
sheet  similar  to  those  in  Figures  3 and  4 of  this  addendum. 

The  status  table  (Figure  3)  shall  keep  a running  summary  of 
all  revisions.  Specific  details  shall  be  recorded  on  the 
revision  sheet  (Figure  4) . Revisions  to  a Test  Design  and 
Instruction  Document  shall  also  be  indicated  by  tagging  each 
revision  page  with  a revision  letter  in  the  upper  right-hand 
corner  (for  example.  Revision  B) . A capital  letter  shall  be 
used  to  tag  an  added  page  (for  example,  page  3A,  3B,  etc.). 

Table  of  Contents.  Section  2.1  illustrates  the  high-level  table 

of  contents  for  a Test  Design  and  Instruction  Document.  Further 

breakdown  of  the  sections  shall  be  required. 

Section  1 - Introduction.  This  section  shall  contain  the  following 

information  in  the  order  indicated. 

••  UUT  Description.  Provide  a brief  physical  and  functional 
description  of  the  UUT. 

••  UUT  Differences.  If  more  than  one  UUT  is  to  be  covered  in  the 
same  Test  Program  Design  Document,  the  differences  between 
UUTs  shall  be  noted  by  the  use  of  Army  Part  Numbers  and/or 
serial  number  ranges.  The  test  flow  diagram  shall  be  designed 
to  reflect  these  differences. 

• • Purpose  and  Scope  of  Test.  Describe  the  scope  of  the  test 
program  as  well  as  the  overall  test  philosophy  and  required 
fault-isolation  levels. 

••  Programming  Rules.  The  basic  programming  rules  for  the  desig- 
nated ATE  shall  be  contained  in  its  Programming  Manual.  The 
manual  shall  contain  general  information  on  the  operating 
characteristics  of  the  ATE  assemblies,  UUT  interface  connection 
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data,  and  detailed  procedures  applicable  to  the  ATE  test  analy- 
sis and  program  structure.  State  the  test  program  language 
called  for  by  use  of  the  designated  ATE. 

••  Source  Information.  List  sources  of  information,  documents, 
drawings,  etc.,  used  in  the  preparation  of  the  Test  Design  and 
Instruction  Document. 

• Test  Description  and  Instructions 

• • Pretesting  Data 

• • • ATE  Resources  Required.  List  by  assembly  number  (with 
title  and  quantity  of  all  ATE  resources  required  to  run 
the  test  program) . 

• • • Support  Equipment  Required.  List  by  nomenclature  and 

number  any  test  equipment  tools,  fixtures,  shorting  plugs, 
etc.,  required  in  addition  to  the  ATE  station  and  ID  (In- 
terconnection Device) . Do  not  list  standard  tools  in  the 
Test  Design  and  Instruction  Document. 

••  Test  Requirements  Analysis.  This  section  shall  contain  the 
material  required  by  Subsection  2. 3.1.1  of  the  Guidelines  for 
Acquisition,  Use,  and  Configuration  Control  of  TPSs  for  ATE. 

• • Program  Start  Procedure.  Provide  step-by-step  procedures  for 
all  unique  actions  of  the  test  program  required  to  start  the 
test  (i.e. , entering  test  program's  call-up  code).  The  proce- 
dures shall  be  on  the  basis  of  the  condition  that  the  ATE  sta- 
tion is  properly  set  up,  turned  on,  and  ready  for  UUT  (Unit 
Under  Test)  testing.  If  unique  actions  are  not  required  for 
start  of  testing,  enter  program  number  only.  Set-up  illustra- 
tions shall  be  provided,  if  required,  by  the  complexity  of  the 
buildup,  which  may  be  two-  or  three-dimensional. 

• • Operator  Actions.  Testing  data  included  in  the  Test  Design  and 
Instructions  consist  of  only  adjustment  and  alignment  proce- 
dures , instructions  for  operator  intervention  at  the  interface 
(except  removal/replacement) , and  other  messages  that  cannot  be 
displayed  on  a single  CRT  (Cathode  Ray  Tube)  display  frame  or 
other  display  devices,  illustrations  (i.e.,  waveforms,  sketches, 
etc.),  and  references  thereto.  References  to  the  testing  data 
shall  appear  on  the  display  panel.  Preferably,  illustrations 
referenced  will  be  placed  on  the  same  page  as  the  testing  data. 

Illustrations  of  the  UUT  with  all  operator  actions  points  should 
be  provided  as  required  and  should  show  the  reference  designa- 
tion on  or  near  the  components  involved.  The  only  components 
that  shall  be  illustrated  are  those  necessary  for  positive 
identification  of  the  operator  action  points.  If  there  are 
no  operator  action  points,  no  illustration  should  be  provided. 

Any  safety  points  shall  be  identified  with  a warning  on  the 
illustration.  Illustrations  shall  be  provided  to  show  where 
alignment/adjustment  components  are  located. 
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• • English-Language  Test  Description 

•••  General . This  section  shall  contain  a functional  flow 
chart  prepared  in  accordance  with  the  ATE  Programming 
Manual  and  description  of  each  test.  A description  of 
the  functional  tests  shall  be  included.  The  purpose  of 
this  description  shall  be  intended  to  (1)  summarize  the 
test  functions  and  the  test  numbers  associated  with  each 
test  function,  (2)  discuss  the  test  philosophy  in  suffi- 
cient detail  to  point  out  the  parameters  of  the  UUT  that 
are  tested  with  reference  to  the  functional  test  number 
involved,  and  (3)  discuss  functional  flow  diagramming  in 
order  to  make  this  section  of  the  flow  chart  clear  to  a 
reader  unfamiliar  with  ATE  application  test  programming. 

• • • Test  Design  Criteria.  This  section  shall  contain  all 

rules  that  apply  to  the  TPS  contractor  in  TPS  generation. 

In  general , the  design  criteria  shall  be  contained  in  the 
Guidelines  for  Acquisition,  Use,  and  Configuration  Control 
of  TPSs  for  ATE. 

•••  Static  Tests.  This  section  shall  describe  all  static 

tests  conducted  prior  to  application  of  power  for  fault- 
isolation  purposes. 

•••  Dynamic  Tests.  This  section  shall  describe  all  tests 

conducted  to  prove  that  the  UUT  is  operating  within  its 
performance  specification. 

•••  UUT  Interface  Requirements.  For  each  UUT,  the  contractor 
shall  prepare  an  ATE  UUT  interconnection  diagram. 

— Necessary  interface  between  the  ATE  functions  through 
the  patch-board  to  the  UUT  and  through  the  test  acces- 
sories shall  be  provided  by  the  contractor  or  agency 
responsible  for  program  validation. 

— Tests  requiring  accuracies  or  capabilities  not  available 
in  the  ATE  shall  be  documented.  Where  active  signal 
conditioning  is  required  in  the  adapters  to  provide  UUT 
test  capability  not  available  in  the  ATE,  the  require- 
ments shall  be  determined  by  the  contractor's  good 
engineering  practices  and  shall  be  specified. 

— Layout,  fabrication,  and  static  checking  of  each  adapter 
and  special  adapter  cable  shall  be  accomplished  by  the 
contractor.  Changes  to  the  interconnection  diagram  as 
a result  of  adapter  layout  shall  be  provided  by  the 
contractor. 

— A proposed  spares  list  for  the  ID  shall  be  provided. 

••  Supplementary  Data.  Following  is  a brief  description  of  all 

elements  of  supplementary  data: 

•••  Listing . Provides  the  complete  source  listing  (including 
ID  data)  of  the  test  program  and  shall  consist  of  a 
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standard  80/80  listing  of  the  source  deck.  Since  most 
programs  are  rather  voluminous,  it  is  strongly  suggested 
that  the  submittal  of  the  program  for  listing  be  performed 
once  when  all  other  reports,  tables,  and  flow  charts  have 
been  coded  and  debugged. 

• • * Functional  Flow  Chart  (FFC) . Provides  the  end-to-end 
UUT-oriented  philosophy  of  the  UUT  on  a major  function 
basis.  This  programmed  flow  chart  is  intended  to  provide 
the  user  with  an  overall  ("big  picture")  test  philosophy 
of  the  UUT.  Its  content  should  generally  consist  of  end- 
to-end  tests,  sequenced  in  the  order  performed. 

•••  Diagnostic  Flow  Chart  (DFC) . Provides  the  detailed  UUT- 
oriented  methods  employed  on  a test-by-test  basis  showing 
all  branching  and  including  the  purpose,  methodology,  and 
expected  results  of  each  test. 

•••  Interconnection  Diagrams.  Provides  the  UUT,  ID,  and  ATE 
interconnection  information  test  by  test.  Two  types  of 
diagrams  are  required,  a system  diagram  and  test  diagram. 

•••  Cross-Reference  Table.  If  digital  automatic  test  genera- 
tion is  employed  in  the  TPS,  all  inputs  and  outputs  of  the 
digital  simulation  technique  shall  be  provided,  including 
any  design  aids  used  by  the  TPS  contractor. 

Engineering  Notes.  Any  special  UUT  or  information  that  should  be 
brought  to  the  attention  of  the  operator  shall  be  included  in  the 
TPS,  for  example,  telling  the  operator  that  some  particular  action 
may  result  and  shall  be  expected.  Notice  of  such  items  should  be 
limited  to  those  that  might  cause  the  operator  to  unnecessarily 
terminate  testing.  Descriptive  and  theory  information  on  the  UUT 
or  test  program  shall  not  be  included. 

Quality  Assurance  Provisions 

••  Legibility.  The  original  and  all  copies  of  this  document  shall 
meet  industrial  practices  for  legibility.  Standard  methods  of 
reproduction  may  be  used  provided  that  legibility  is  retained 
for  all  copies. 

••  Manuscript  Preparation.  The  initial  pages  of  the  Test  Design 
and  Instruction  Documents  shall  have  formats  similar  to  those 
shown  in  Figures  1 through  4 of  this  Addendum.  Test  pages  shall 
be  typewritten  and  shall  be  numbered  consecutively,  using  Arabic 
numerals.  Tests  in  the  flow  chart  may  be  hand-lettered.  Sche- 
matic and  interconnecting  diagram  lettering  may  also  be  prepared 
free-hand.  For  logic  symbols,  the  ANSIX3 . 5-1970  or  equivalent 
must  be  used. 

••  Hazard  Notice.  Hazards  associated  with  the  tests,  together  with 
appropriate  warnings  or  cautions,  shall  be  included  in  the  Test 
Design  and  Instruction  Document.  If  there  are  no  hazards  asso- 
ciated with  the  testing,  a statement  to  that  effect  shall  be 
inserted  in  Section  2.8,  Engineering  Notes. 


Figure  1.  SAMPLE  TITLE  PAGE 


••  Testing  Flow  Charts.  Flow  charts  for  testing,  in  conjunction 
with  Program  Listings,  shall  depict  all  program  steps.  The 
actual  message  to  appear  either  as  instruction,  information, 
or  in  a printout  shall  be  included. 
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Block  1.  Test  Program  Set  Design  Plan 

Block  3.  This  plan  is  used  by  the  procuring  activity  to  assess  the  con- 
tractor's planning  for  the  TPS  computer  programs,  interface 
device,  and  documentation;  to  approve  the  contractor's  approach 
for  TPS  development;  and  to  monitor  and  evaluate  the  contractor's 
effort  in  TPS  development. 

Block  7.  The  DID  applies  to  all  TPS  development  and  acquisition  contracts. 
The  plan  may  be  obtained  during  the  proposal  phase. 

Block  10.  The  TPS  Design  Plan  is  a document  identifying  the  contractor 

efforts  required  to  develop  and  deliver  the  TPSs  and  any  neces- 
sary TPS  maintenance  hardware  and  software  in  accordance  with 
the  terms  of  the  contract.  As  a minimum,  the  plan  shall  include: 

• The  organizational  responsibility  and  structure  of  the  group 
that  will  be  designing,  producing,  and  testing  all  TPSs 

• The  management  and  technical  control  that  will  be  utilized 
during  TPS  development 

• The  methodology  for  ensuring  satisfactory  TPS  design  and 
testing,  including  measures  necessary  to  ensure  testability, 
criteria  for  adequacy,  and  feasibility  of  requirements;  to 
establish  compliance  of  the  test  plans  and  procedures;  to 
monitor  tests  and  certify  test  results  and  test  reports;  and 
to  ensure  maintenance  of  test  documentation 

• The  approach  that  will  be  taken  to  develop  and  qualify  the 
elements  of  the  TPS 

• The  schedule  allocated  for  the  development  of  each  TPS,  the 
schedule  and  procedures  for  preparation  and  execution  of  the 
concept,  detailed  design  reviews,  and  the  acceptance  test 

• The  approach  for  monitoring  and  reporting  the  progress  and 
status  of  TPS  development 

• The  level  of  manpower  that  will  be  allocated  to  each  TPS  devel- 
opment task  beginning  with  preparation  of  the  initial  Test 
Design  and  Instruction  Document  and  the  ID  Development  Specifi- 
cation through  qualification  testing  of  each  TPS 
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• The  contractor's  capability  or  plan  for  establishing  the 
development  of  TPS  computer  programs  and  the  controls  that 
will  be  implemented  to  ensure  availability  of  the  data 
processing  equipment  necessary  for  verifying  the  software 

• The  extent  of  in-plant  testing,  including  test  planning,  test 
methods,  documentation,  and  test  reporting 

• The  general  procedures  for  reporting,  monitoring,  and  resolv- 
ing TPS  program  errors  and  deficiencies  during  in-plant  test- 
ing, corrective  actions  will  include  analysis  of  problem  and 
deficiency  reports,  analysis  of  trends,  and  methods  of  ensur- 
ing adequacy  of  corrective  measures  taken  and  their  proper 
implementation . 

• The  techniques  that  will  be  used  for  controlling  the  design  of 
the  TPS  and  for  ensuring  that  all  performance  and  design  re- 
quirements have  been  implemented  within  the  intent  of  the  sys- 
tem, these  techniques  will  include  the  controls  necessary  to 
ensure  that  the  design  of  the  software  follows  the  requirements 
of  the  Test  Design  and  Instruction  Document  and  that  the  ID 
design  follows  the  development  specification. 

• The  approach  for  configuration  control  of  computer  program 
master  tapes  and  associated  documentation,  including  identifi- 
cation of  both  source  and  object  code  in  various  forms  or 
versions  from  initial  approval  or  acceptance  until  incorporated 
into  the  final  deliverable  medium,  the  configuration  manage- 
ment procedures  will  be  audited. 

• The  identification  of  special  simulation,  data  reduction,  or 
utility  tools  that  are  planned  for  use  in  the  development  of 
the  TPSs  that  are  not  deliverable  under  the  terms  of  the  con- 
tract. In  addition,  those  tools,  techniques,  and  methodolo- 
gies used  to  support  the  objectives  of  the  program  will  be 
identified,  and  a description  of  their  use  to  augment  or 
satisfy  program  requirements  will  be  provided. 

• The  plan  for  ensuring  TPS  growth,  modularity,  and  ease  of 
modification 

• The  approach  for  developing  TPS  documentation,  including  stan- 
dards used,  provisions  for  ensuring  delivery  of  correct  docu- 
mentation and  change  information,  and  provisions  for  periodic 
review  by  an  independent  agency  and  designated  approval 
authority 

• The  approach  for  ensuring  TPS  and  ATE  compatibility 
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Block  1.  Test  Program  Set  Software  Media 

Block  3.  This  plan  is  used  by  the  procuring  activity  to  define  preparation 
requirements  for  TPS  computer  software. 

Block  7.  These  preparation  requirements  apply  to  all  TPS  software  developed 
for  the  UUT. 

Block  10.  The  TPS  software  media  requirements  shall  consist  of  but  are  not 
limited  to  the  following: 

• Preparation  for  delivery  shall  be  in  accordance  with  the 
contract. 

• Source  programs  shall  be  delivered  as  specified  on  DD  Form  1493 
(e.g.,  on  punched  cards,  magnetic  tape,  or  other  specified 
medium)  using  the  Government-approved  language.  The  source 
form  shall  be  suitable  for  assembly  or  compilation,  as 
appropriate. 

• Object  code  shall  be  prepared  in  machine  language  and  delivered 
as  specified  on  DD  Form  1493  (e.g.,  on  punched  cards,  magnetic 
tape,  or  other  specified  medium) . The  object  code  form  shall 
be  suitable  for  loading  and  execution  in  the  specified  automa- 
tic test  equipment  (ATE) . 
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TPS  ACQUISITION  CHECKLIST 


The  TPS  checklist  is  provided  to  facilitate  the  use  of  the  guidelines 
provided  in  this  document  for  the  preparation  of  a technical  statement  of 
work  (SOW) . It  lists  the  type  and  quantity  of  data  to  be  specified  for  the 
specific  TPSs  that  will  be  procured.  Each  data  item  is  cross-referenced 
with  the  section  number  of  this  document  in  which  detailed  descriptions  or 
guidelines  are  provided. 


Section 


Required  Data 


2.1 


2.3.1 


2. 3. 5. 2 


2. 3. 5. 3 


2.4.2 


2.4.2 


2.4.2 


2.5 


2.5.4 


2.6.1 


Define  the  UUTs  and  ATEs  and  their  characteristics  (language, 
configuration,  version,  etc.). 

Identify  the  data  items  and  their  sources  and  status  (prelimi- 
nary, final,  etc.)  on  UUT,  ATE,  and  ATE  support  software. 

Inform  the  contractor  of  the  security  classification  of  the 
UUT,  ATE,  and  TPS  data  and  the  applicable  security  regulations. 

State  the  period  of  warranty  required  for  the  TPS. 

State  the  intended  usage  environment  of  the  TPS  (factory, 
general  support,  or  direct  support). 

Specify  if  ID  has  to  be  fully  militarized. 

Specify  if  MIL-STD-471  is  to  be  used  for  ID  maintainability 
demonstration. 

Specify  performance  and/or  diagnostic  testing  requirements  and 
level  of  isolation  (to  an  LRU,  a card  or  a number  of  components, 
etc.).  Specify  environmental  testing  requirements. 

Specify  environmental  conditions  under  which  TPSs  must  be 
tested. 

Specify  the  medium  to  be  used  for  delivery  of  the  TPS  source 
and  object  code  and  support  software  programs. 
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ORGANIZATION  OF  TPS  OPERATIONS 


A centralized  TPS  support  engineering  organization  is  recommended, 
which  should  consist  of  a TPS  Organic  Maintenance  Group  and  a TPS  Analysis 
Group  (see  Figure  F-l). 

m ~ 

1.  OPERATOR  FUNCTION 

» - 

The  primary  function  of  the  operator  is  to  test  and  diagnose  the  UUTs 
and  to  perform  limited  maintenance  on  the  ATE.  The  operators'  maintenance 
responsibilities  may  vary  slightly  at  the  factory,  depot,  general  support, 
and  direct  support  levels.  However,  the  maintenance  responsibility  of  the 
operators  shall  be  limited  to  the  following: 


• Maintenance  in  accordance  with  preventive  maintenance  schedules 

• ID  maintenance  whenever  possible;  sufficient  sets  of  ID  spares 
shall  be  included  in  the  Essential  Items  Stockage  List  (EISL) 

• ATE  instrument  or  module  replacement  only 
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Operator  diagnostics  shall  be  limited  to  those  functions  automatically 
directed  by  the  ATE.  The  operator  shall  be  provided  with  intermediate 
instructions  and/or  object  code  for  TPS  execution  and  shall  not  be  per- 
mitted to  perform  software  maintenance  or  program  enhancement  and  develop- 
ment. Figure  F-2  presents  an  overview  of  ATE  operations. 

1.2  Production  Test  Scheduling 

On  the  basis  of  the  complexity  and  peculiarities  of  the  UUTs  being 
tested,  a production  test  schedule  shall  be  prepared  for  each  test  station 
to  minimize  set  up  and  tear  down,  as  well  as  ATE  self-test  and  calibration 
requirements. 

For  stations  testing  many  different  types  of  UUTs,  a computerized 
scheduling  system  to  optimize  ATE  and  TPS  usage  is  highly  recommended.  In 
addition  to  scheduling  the  testing  of  UUTs,  the  scheduling  schemes  will 
select  and  sequence  the  many  types  of  tests  and  retests,  possibly  by  UUT 
serial  number,  other  identification,  or  by  criteria  derived  from  previous 
UUT  test  history. 
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Maintenance 


Figure  F-2.  OVERVIEW  OF  ATE  OPERATIONS 
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1 . 2 ATE  Logs 
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All  ATE  activities  shall  be  entered  and  kept  current  in  a log  book 
for  each  ATE  station.  Log  book  entries  shall  include  all  ATE  and  ID 
maintenance  (corrective  and  preventive)  actions,  operator  sign-in  and 
sign-out,  and  a record  of  all  UUT  testing  activities.  Any  anomalies 
observed  during  the  tests  shall  be  logged  and  reported  to  the  TPS  Analysis 
Group . 

For  TPSs  providing  adequate  UUT  testing  records,  the  printout  from 
the  print  device  shall  furnish  the  necessary  UUT  testing  data,  and  an 
additional  entry  into  the  log  book  for  testing  data  will  not  be  necessary. 
When  possible,  a copy  of  the  UUT  testing  data  shall  be  provided  on  a 
magnetic  tape  for  maintenance  analyses  and  reporting  purposes.  If  magnetic 
tape  logging  is  unavailable,  it  is  recommended  that  multi-layer  papers  be 
used  in  the  ATE  print  device  to  provide  an  additional  copy  for  maintenance 
purposes. 

2.  TPS  MAINTENANCE 

TPS  maintenance  consists  of  the  following: 

• Correction  of  errors  discovered  in  the  TPS 

• Enhancements  to  the  TPSs  to  improve  efficiency,  operator  interface, 
or  utility 

• Modifications  to  the  TPSs  because  of  changes  in  the  UUT,  ATE,  or 
support  software  and  equipments 

The  TPS  Analysis  Group  is  the  only  authority  to  approve  TPS  maintenance 
actions.  This  group  will  serve  as  a single  point  of  contact  with  Army 
Product/Item  Managers,  configuration  control  boards,  and  TPS  users  at  all 
locations  using  the  TPSs  supported  by  the  group. 

Requests  for  TPS  changes  are  reviewed  or  initiated  and  analyzed,  and 
a course  of  action  is  recommended  by  the  group.  If  a TPS  change  is  approved, 
a determination  will  be  made  to  implement  the  changes  through  warranty 
applications,  contractor  maintenance  service,  or  organic  TPS  maintenance. 

Upon  implementation  of  the  proposed  changes,  the  TPS  Analysis  Group  shall 
perform  the  necessary  evaluations,  assign  revision  numbers,  and  release 
them  for  production  use.  All  TPS  maintenance  implementations  shall  be  in 
accordance  with  the  guidelines  stated  in  this  document  for  new  TPS  acquisi- 
tion. The  systems  analysis  personnel  shall,  at  the  time  of  approving  a 
change,  determine  the  types  and  the  extent  of  testing  that  shall  be  neces- 
sary following  implementation  of  the  TPS  change.  In  all  cases,  the  mainte- 
nance action  shall  include  a complete  update  of  all  TPS  documentation  as 
defined  in  this  document. 

2 . 1 Warranty  Coordination 

The  TPS  Analysis  Group  shall  determine  if  the  required  changes  are 
covered  under  the  terms  of  any  warranties  or  maintenance  contracts  and 
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take  the  appropriate  actions,  either  directly  or  through  the  procuring 
agencies  to  request  corrective  actions. 

2 . 2 Organic  Maintenance 


The  TPS  Organic  Maintenance  Group  provides  the  Army  with  the  in-house 
capability  of  performing  TPS  maintenance.  Depending  on  personnel  avail- 
ability and  the  type  and  extent  of  TPS  maintenance  required,  organic 
maintenance  is  recommended.  Small  maintenance  actions  that  do  not  warrant 
documentation  for  outside  contracting  and  those  changes  requiring  close 
coordination  between  the  various  Army  maintenance  organizations  shall  be 
performed  organically.  Organic  maintenance  capability  may  be  augmented  by 
contractor  personnel  working  under  the  direction  of  Army  organic  mainte- 
nance as  an  extension  of  Army  resources  during  periods  of  high  demand  for 
maintenance. 

2 . 3 Contractor  Maintenance 


For  extensive  changes  or  rewrites  of  the  TPSs,  contractor  maintenance 
is  recommended.  Acquisition  of  contractor  maintenance  services  shall  be 
treated  in  the  same  manner  as  procurement  of  new  TPSs,  where  applicable. 
Reviews,  audits,  and  validation  testing  of  TPSs  that  have  undergone  major 
rewrites  or  extensive  changes  shall  be  identical  to  those  required  for  new 
TPS  acquisition. 

3.  ANALYSIS  REQUIREMENTS 

The  minimal  analyses  necessary  for  approval  of  TPS  changes  shall  con- 
sist of  the  following: 

• Effect  of  the  proposed  change  on  operational  aspects  of  other  TPSs 

• An  economic  or  cost-effectiveness  analysis  for  those  changes  that 
are  intended  to  enhance  the  TPS 

• Effect  on  testing  schedules 

• An  analysis  to  determine  the  possibility  that  changes  can  be  made 
to  the  software  portion  of  only  the  TPS  and  not  to  the  interface 
device;  ID  changes  normally  require  a long  lead  time 

• A reliability  and  maintainability  analysis 

• A review  of  historical  data  on  the  types  of  changes  requested 
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